Case Study Outlines

10/5/99

Rev. 19/06/00
Agora case study proposal: Bath Spa University College
May 2000

As the end of semester is approaching, it was though more useful to carry out a case study involving the library staff. The proposal has the principal aim of determining the training needs of (i) those who are familiar and (ii) those who are generally IT literate but unfamiliar with online services. Another, lesser aim is to determine the kind of documentation best suited to helping both types of user.

The library staff fall into two broad groups. The information assistants are familiar with automated systems for circulation, acquisition, etc. but considerably less so with searching either bibliographical databases or online services for information. The professionally qualified staff are familiar both with library systems and with online sources which they also teach to both undergraduate and postgraduate students and to academic staff.

· Library staff will be introduced to the Agora interface as part of our Friday morning staff training sessions.

· They will then be asked to carry out tasks, within a given timescale, eg. an hour, concerned with identifying resources on a particular topic.

· A smaller group will then be shown the landscape function and asked to construct a suitable landscape for use by a particular subject group, eg. geography or psychology.

· Staff will be given a questionnaire and interviewed about their experiences and asked what further help would have been useful or how they would have tackled these tasks when teaching students.

· Finally, if time and schedules allow, staff will be interviewed as a group about their experiences.

Ann Siswell

Head of Library Systems

Bath Spa University College

8 May 2000

AGORA CASE STUDY AT THE UNIVERSITY OF HULL

Introduction

The University of Hull is a medium-sized university with 14,722 students including 1,578 research postgraduate students and 1,786 taught postgraduate students. Students are split across five faculties (Arts, Engineering and Mathematics, Health, Science and the Environment, Social Sciences) and the Institute for Learning with Arts and Social Sciences forming the largest faculties.

The Library is part of Academic Services, a converged service with responsibility for all academic and administrative computing, library services, aspects of learner support and a wide range of activities concerned with the use of technology in learning and teaching. The experience of the Academic Services Training Team which brings together trainers from the Library, Computing and Learning Development and which has developed a comprehensive training programme covering C&IT and information-handling skills will be of particular relevance during the evaluation phase of Agora.

As a member of the RIDING consortium, library staff at Hull are already familiar with the VDX web interface and further evaluation of the RIDING gateway by end-users which will be taking place during the evaluation phase of Agora may provide additional input to the case study.

The University of Hull case study will compare the views of two different user groups (academic/research staff and taught postgraduate students) on the benefits of cross-domain searching in general and on the advantages and disadvantages of AGORA in particular. In addition UoH will ascertain the views of library staff on the usefulness of cross-domain searching and on the implications for user education and enquiry desk work.

Methodology

In early July Agora will be demonstrated to a small group of library staff and users to introduce them to the concept of cross-domain searching and to explain the project. Academic staff and postgraduates will be offered four free document supply requests as an incentive to participate in follow-up interviews. The following staff will be included:

1 academic/research staff from the Faculty of Arts

1 academic/research staff from the Faculty of Social Sciences

1 taught postgraduate from the Faculty of Arts

1 taught postgraduate from the Faculty of Social Sciences

2 members of Academic Services staff from the Training and Documentation team

2 members of Academic Services staff working on the Enquiry Desk

The follow-up interviews with staff and students will be based on a questionnaire which will be completed as each participant uses Agora to search for material relevant to a selected topic. The most appropriate landscape will be set up prior to the interview.

The four members of Academic Services staff will be assigned individual usernames and will be asked to spend some time looking in depth at Agora. Interviews will be then be conducted with each member of staff. Academic Services staff will be asked to give their views on the benefits of Agora and cross-domain searching both as an end-user and as an information professional training the end-user to use systems effectively and efficiently. They will also be asked to compare the concept of cross-domain searching (Agora) with cross-catalogue searching (RIDING).

The interviews will be carried out by Bridget Towler, Head of Digital Library Development.

AGORA User Case Studies

Draft outline proposal from Heriot-Watt University Library.

Introduction

Heriot-Watt University is essentially a technological university, with also a strong interest in business, management, economics, languages and textiles. It is based on two campuses, one at Riccarton on the outskirts of Edinburgh, and one at Galashiels in the Scottish borders. As well as strong undergraduate and postgraduate programmes in these areas, 45% of Heriot-Watt's income comes from the research, training and commercial services provided to the commercial sector.

Enhancing the provision of information services to its community is a high priority for the Library, and hence its participation in this project.

The case study

Heriot-Watt University Library would like to focus on looking at the effectiveness of the ‘landscape’ concept to the work of academics and post-graduate students. In particular, the effectiveness of structured landscapes vs. unstructured landscapes. We wish to explore how applicable the ‘landscape’ concept is to the work of non-library staff, whether they can understand what it is possible to achieve by applying the concept, and whether it can help in the teaching and setting of assignments, compared with the methods used currently for finding information to set as appropriate sources of information for students to use. There may be outcomes, which we cannot envisage at present.

The case study will be carried out by Shirleen Craig (Faculty Librarian for the School of Management), who has been representing Heriot-Watt University in this project, with assistance from colleagues in Heriot-Watt’s Institute for Computer Based Learning, who are hosting the Agora server for us.

Methodology

It is envisaged that the study will be carried out using a small base of interested academics from different subject areas and some students connected with them (however, this is with the caveat that it might be difficult to locate suitable postgraduate students at this time of the academic year).

Firstly they will be given some training in the ideas behind and use of the Agora system, and then hopefully they will spend some time experimenting in applying it to their work. Regular contact will be kept with them to ensure that they are not experiencing any problems, and to provide any help, but without wishing to interfere with them when this is unnecessary.

The data gathering will take the form of a questionnaire and then a structured interview. although CERLIM will be consulted to help choose the most appropriate methodology.

Shirleen Craig.

AGORA PROJECT UEA CASE STUDY PROPOSAL Vers. 1.0 [Draft]
Date: 05 May 2000 Authors: D. Palmer, K. Inglis

INTRODUCTION & BACKGROUND

The University of East Anglia, as the lead institution within the Agora Project, has a great interest in exploring the possibilities of the Agora HLMS concept, and in creating ‘change management tools’ that will be of use to the wider library community. UEA also has an interest in exploring ways in which the Agora HLMS concept can be ‘transitioned’ into a potential service for the University which will have the maximum beneficial impact for the University.

Thus, UEA has decided to examine the question of interlending within the Agora HLMS Release 1 system. This will examine an issues not addressed by other Library Associates, and therefore provide useful input to the Project. It will provide benefit to the University by offering an opportunity to examine not only interlending within the Agora HLMS, but by extension, an opportunity to review and assess current practices in that functional area.

CASE STUDY SUMMARY

As noted above, the intent is to examine interlending from both a user’s perspective and from the perspective of the library administering the system. In particular, we wish to compare the processing of requests with present practices to assess whether the VDX functionality offers more or less to the Library than the current stand-alone system. From the user’s perspective, we wish to assess the ease with which users can order and track items and the degree to which the integration of discovery and searching effects this process.

We acknowledge the shortcomings of the user interface and will attempt to concentrate the users on assessing the interlending functionality, not the appearance of the system as a whole, or the admittedly ‘quirky’ interface, particularly in the landscapes areas. To that extent we would wish to ‘shield’ the users from the interface by working with them to design landscapes that would act as a ‘default’ for them that would include resources of greatest use. Another option would be simply to bypass the discovery and searching altogether and simply use the blank online form. However, we feel that this might be too restrictive and would negate our desire to assess how integration of discovery and searching effects the interlending process.

We will include additional resources wherever possible that add to the functionality of Release 1 with particular emphasis on resources of interest to our evaluators. It is essential, at an early stage, to ensure that we have resources that are attractive to them and will encourage use of the discovery and search functions to generate requests.

METHODOLOGY

Given our intention to assess both the user and the library community’s reaction to the interlending functionality of the system, we will need to address both groups. Internally, we will concentrate on a small number of Interlending Office staff, knowledgeable in present practices, who will be able to offer a fair and informed assessment of the system. In this respect, one person can act as their own ‘control’ group as it is anticipated that they will continue some regular duties during the case study.

As for the users, we have decided to look at two schools with different needs within the University; the School of Environmental Sciences, and the School of Development Studies. Due to the timing of the case studies, available evaluators can only be drawn from the faculty and postgraduate population. In order to encourage cooperation and a fair trial of the system, we intend to select a small number of people who have, in past, demonstrated use of interlending services and an interest in library matters.

We also plan to select a small ‘control group’ within each school who will be given the same assessment, but for the regular service that the Interlending Office offers. In no way can we assume that either group will be unaware or unaffected by the work of the other, but we hope to gain a balanced view of the service offered by each interlending system.

As to specific methodologies, a combination of interviews, questionnaires and logs will be used. The former two are probably the most effective tool with the users whilst the latter may be the most effective with the library internal assessment of the system. However, at this time, there is no intention to restrict use of methodologies in this way. It is anticipated that consultation will take place with CERLIM staff to help UEA choose and use the most appropriate technology.

TIMETABLE

The timetable for the completion of the case study has certain parameters. The final report must be ready, reviewed and revised by 2 October. It is therefore anticipated in order to draft the various results components of the case study report, the case study will cease in mid-August. Conversely, there will be some time required at the commencement of the case study to design evaluation tools, to select and train the user evaluators, to ensure that the system is in a state appropriate for a proper assessment of interlending, and to train the staff tasked with the evaluation. Therefore, whilst we are aiming for a June 1 start date, we would not be surprised if this date ‘slipped’ to mid-June. This still gives 2 months for evaluation.

AGORA - User Case Study Bridget Robinson May 2000

Proposal for study to be carried out by University of Bath - Division of Access and Continuing Studies(DACS) and UKOLN

Introduction

An opportunity has arisen whereby UKOLN and DACS can provide the resource to undertake a case study concentrating on the requirements of distance learners and their reaction to the Agora HLMS. This will offer a true test of the Hybrid library as a system that is used remotely - away from the traditional bricks and mortar of a physical library.

Background

DACS - The Division of Access and Continuing Studies is made up of 5 Centres:-

· The Centre for Distance Education (CDE)

· Centre for the Development of New Technologies in Learning (CDNTL)

· The Business Services Centre

· The Centre for Access ad Continuing Education

· The Office of Associated Colleges

One of the main areas of growth for the Division as a whole is the area of online learning and the online support of learning - this is especially important in relation to the needs of non-traditional students. CDE and CDNTL are already working with the Royal College of Nursing Institute and The Royal College of General Practitioners in developing CD and web-based support for nurses' and GPs' training respectively. They are also concentrating on developments in "e-universities".

The Centre for Distance Education has expressed a specific interest in Agora in relation to their post-graduate distance learning programmes.

Case Study

The UKOLN/CDE case study will concentrate on the use of Agora by a small group of distance learning post-graduates currently studying on the Construction Management Programme. They will use Agora in the specific area of dissertation work. The ILL module of Agora will be used in a limited way - UKOLN will liase with the library to ensure that procedures are in place to meet student requests.

The study will assess the usefulness of Agora in a distance learning context.

Methodology

The initial introduction to Agora will take place at the dissertation workshop that is being held at the end of June, Agora will be presented to the students as part of the Library training. Students will be offered access to a specific construction landscape that will be put together with the help of the faculty librarian.

Information about the use of Agora will be gathered using questionnaires, journals and telephone interviews, obviously the methods will have to fit in with the work patterns of those participating.

CDE will also provide their own evaluation of Agora as an integral part of distance learning.

AGORA Hybrid Library Management System
Case Study at the University of Edinburgh

Background

Edinburgh University Library is currently undergoing restructuring with a view to enhancing its capacity to develop and support digital library developments. Central to this development is its new Online Services Division. The Director of the Science & Engineering Library, Learning & Information Centre (SELLIC) has from 1 April also become the Sub-Librarian in charge of the Online Services Division. SELLIC now also fits within the new Division, and our involvement in the AGORA Project is expected to benefit from the ability to move from project to full service mode more easily as a result of the restructuring exercise. This should ultimately be of benefit to our use of AGORA.

Case study proposal

We propose running a case study to analyse the use of the AGORA system in the context of a real undergraduate course.

The case study will be hosted by the SELLIC project (the Science & Engineering Library Learning & Information Centre), and will be based in the Faculty of Science & Engineering.

Objectives

The case study will have the following objectives:

· To establish by means of a sample survey how student users rate AGORA according to the following criteria

· ease of use

· speed

· recall and relevance rates

· usefulness of retrieved references

· number of citations found to be useful

· To establish whether the HLMS systems approach offers any advantages or disadvantages over the discrete approach with which students will be more familiar.

Methodology

We wish to work with an undergraduate course in Mechanical Engineering. Students studying either for an Ordinary or an Honours degree in Mechanical Engineering with Management Techniques are required to undertake project work, industrial seminars, group design projects and – for Honours students in the 4^th Year – an individual project including the submission of a report. We expect to work with a total class of around 30 students. The combination of engineering, business studies and management in this course offers a very useful subject for a case study, involving the students in information searching across a range of databases.

The case study will take the following form:

· The SELLIC Director will coordinate the case study.

· The SELLIC Director and the Engineering Subject Librarian will work with the course team to identify the particular piece of project work from those undertaken which will form the most appropriate case study.

· The SELLIC Director and the Engineering Subject Librarian will also work with the course team to establish the range of sources which are likely to be required by students undertaking the project.

· AGORA will be configured to include the target databases required. Those which are Z39.50-compliant will be searchable from a single query. Non-Z39.50-compliant sources will be provided from within the AGORA interface. Sources are likely to include the Library OPAC; Compendex (EDINA); Computer and Information Abstracts (CSA); Inspec (EDINA); Electronics and Communications Abstracts (CSA); Mechanical Engineering Abstracts (CSA); ABI/Inform (ProQuest Direct); Findex (CSA); Web of Science (MIMAS); and European Business ASAP (Infotrac).

· The class will be asked to use the AGORA system in the course of the bibliographic searching required for their assignments. They will be given a survey form to complete as they use the system (this may be delivered to them over the web or by email). Naturally, they will not be compelled to use AGORA, nor to use it exclusively for their assignments.

· The survey results will be evaluated in terms of ease of use, speed and relevance of results. It will also establish which databases they used, the amount of time spent in searching databases and locating relevant articles and reports, the rates of recall and relevance from each search, the references followed up for their projects and the number of citations used in the project which originated in each database. They will be asked to rate the AGORA service in comparison to their perception of their ‘normal’ method of conducting an information search.

· At the conclusion of the project, SELLIC and library staff will conduct a brief group interview with the class, to present the quantitative results and to ask for their comments on AGORA. Key themes emerging from this qualitative feedback session will be reported.

· A report will be written and submitted to the AGORA Project team. With permission from the AGORA team, it may also be submitted to the professional press for publication.

John MacColl. 26 May 2000.