Reference Librarians' Page

Here are the plans / goals we set for increasing our assessment of student learning outcomes achieved by the library at the December retreat. Much of this is in Heather’s notes - but there is a little more detail here.

REMINDER - What we want to assess is Student Learning - not our performance

First - here is the list of assessment that we were already doing.

1. Collected, and sometimes graded, Exercises from library instruction sessions for courses - eg. Education, Art, Biology, Psychology, and FP

2. Librarian observations during the hands on portions of workshops we give - also observations from reference desk inquiries we get from students who attended those workshops - we can often tell how much they did or did not retain from the session or which parts they did retain.

3. Informal feedback from professors after library instruction sessions for their courses - can be in person at the time, in person later, or via email

4. Usage statistics from databases and other online sources and tools
5. Circulation statistics - including in-house use - shows how much using resources, and can also see if using specific resources demonstrated at sessions.
6. Interlibrary loan statistics and data including both Illiad and I-Share

7. Informal feedback from students - verbal or email

8. Gate count

9. Acquisitions Data - can tell us about how well we are meeting goals of providing resources

10. Instruction Data - Data on how many sessions, what subjects, etc.

11. FYILLAA Data - We have 3 sets of data - 2 years of pre-test and 1 year of post test (after the 1st pre test)

Here is what we said we would try to start doing Winter Term

1. Continue to & increase where possible collecting exercises from instruction sessions - try to do this more deliberately, and record or summarize the data. So, even if not graded for professor, we could collect & assess & write up a summary for each and share/store this data. One issue - we often want students to keep the hands on exercises as they become their handout for future reference OR have actual results they need for their individual course projects. Possible solutions - a)we could just not collect these, but try to make observations during the session and make notes quickly afterwards so that we don’t forget b) we could collect while they were doing the one minute assessment and then copy and hand back c) we could collect, assess and then quickly return the next day.

2. Continue and increase making observations during hands on portions of workshops - We would try to do this more consciously, to make notes quickly after the session, and record/share the info.

3. Continue to receive & augment collection of feedback from professors, and try to record/store this data so that it can be added into other data and/or shared. Institute a more formal/semi formal survey or inquiry to professors who have had instruction sessions at the end of the term - did they find the session helpful, did their students have good bibliographies or whatever would be appropriate for that class.
4. Start doing a Quick immediate student survey after sessions

5. Review and collate ILL Data to ascertain connections to instruction - eg could look at things like % of requests for things we own - is it going up or down - is it different for courses with instruction and those without. Can look at # of requests for courses with or without instruction. Could look at if using SFX with/without.

6. Augment collecting info form Reference desk inquiries - do we want to count different things or keep any kind of notes. Could keep a notebook at the reference desk (or page on blog) and make notes when encounter students when it becomes clear they have been at instruction sessions. Or we could add a column to stats for if Question from student who had been at an instruction session - idea that if asking at desk learned at least something from the session & now learning more.

7. Continue to collect database and online tool usage stats, including libguides - We could figure out a more systematic way to do this and what we want to report, how often, and maybe figure out a way to divide up the task or who to assign it to.

8. Get some feedback on Moodle use - Can we get info from Vicky or a survey of professors who are linking to resources via Moodle.

9. Continue to collect and collate Instruction Data - Do this on a term by term basis rather than annually. Do we want to add anything to what we are collecting?

10 Review student papers with bibliographies - target specific classes or ask all professors in hopes of getting some? Look at both those with sessions and without or just those with sessions?
11. Create our own student exit survey - It is not clear from my notes if we meant when they graduate or at the end of each year or at the end of the term when they’ve had an instruction session

12. Pre Test / Post Test - again neither my or Heather’s notes are clear if this is per session or per entry / graduation. Could do either or both.

I also have something in my memory, but not in my notes or Heather’s, about doing something where we got names of students who had attended instruction sessions and then used that to create groups of those who had attended and those who had not and then use that to compare to other data - GPA, other surveys, ILL data etc.

So - Decisions we need to make / Action we need to take :

1. A lot of these things are adding to recording observations and informal feedback, etc. and doing it on a class by class / term by term basis.  Do we want to set a “due date” and format for reporting these things?

2. For #1 on task list - Do we want to collect hands on exercises from more sessions? Do we just want to encourage those who are doing and/or willing to do this to write up a summary for each class they do this in at the end of each term?  For #1 & #2 Do we want to establish some specific questions to address in a summary that folks would record at the end of each term.
3.  For #3 Professor Feeback - do we want to

a) Have everyone agree to ask for some feedback from professors, in whatever mode they want at the end of the term for professors they did sessions with

b) Create a survey for professors that we would each send to those we’d done sessions with at the end of each term

c) Do either a) or b) above, but just with some of our sessions instead of all

4. For #5 - do we want to set up a project for dumping and analyzing ILL data - if so when, and who and when would goal to complete be.

5. For #6   - do we want to add categories to our stats sheet and/or keep a notebook - one notebook or individually - or do we just want each person to be more conscious of these things and to write up a summary of reflections/observations as part of their instruction report at the end of each term?

6.  For #7 - do we want to define specific points of usage data that we want to collate/record regularly and if so when/who/when report

7.  For #8 - Should we ask Vicky about Moodle info - or do a survey of faculty - just sent to those using moodle (list from Vicky)

8.  For #9 - Do we want to add to what we report/record about instruction on a term by term basis

9. For #10 - how do we want to try to get access to student papers with bibliographies

10.  For #11 - do we want to do an exit survey - if so how/when/who

11.  For #12 - do we want to do pre/post tests - for sessions, for years for matriculation/graduation? or both - if so how/when/who

The notaries that I know of on campus are:

• Vicky Trant, Business Office
• Becky Canfield, Human Resources Office

Add knowledge of other notaries in the comments.

The bookstore has a fax machine for student use.

I think we need another long meeting this Wednesday.  We will probably need to have mostly long meetings this month in order to have the decisions on the databases we renew, or purchase through CARLI made by the end of the month.

Laurie has updated the database spreadsheet in the Library/Reference/Databases folder with current pricing from the CARLI list.  She has also indicated the databases that we need pricing from Anne for - to add to that spreadsheet - and this may already have been done or be in process.

Laurie has also sent us the info to link to the whole list to.

So, we need to decide what we want to do at this next meeting and what we want to have done before it :  )

To start this process, here is a suggestion for how we might want to start moving on this for the next meeting.  Please comment and suggest changes or alternatives by the end of the day Tuesday.  Then I’ll send out a more finalized agenda on Tuesday night.

• My suggestion is that we start with our current databases, the ones that we have as “review each year” and the ones that we want to consider platform changes to.  So, we should all look at Laurie’s spreadsheet before the meeting, identify these and think about them.  People would then come the meeting ready to discuss and/or “vote” on these.

• If at all possible I suggest that Anne and Sharon make sure they have contacted Art & Music (and Ed if Anne was still going to ask about this too) so that we can finalize the decision about switching platforms for those.

• Sharon should complete the draft spreadsheet identifying Budget Line, Invoice responsiblity & management responsibility for databases, Database like periodical package subscriptions, and Subscription based/database like Reference sources.

• I recommend that we hold off on the non CARLI list related stuff for now (ie Rankings).

• I recommend that we use this first meeting just to look at existing databases & changes, and then use the next meeting to talk about possible new additions.

This is meant to just be a suggestion - not an agenda.  So, please comment, suggest changes, add your own ideas etc.

Each librarian should add a comment listing and ranking the databases/services/resources that he or she thinks should be listed in the general category of the Subject Databases list.

Please attach as comments a ranked list of the databases/services/resources for each of the subjects you are liaison to.

Attending - Sharon, Laurie, Anne & Scott (Jeff out of town at IACRL)

Subs - nothing new

Instruction / Reference - Break so nothing

Databases

Specific Databases :

International Index of the Performing Arts (Chadwyck-Healy/Proquest)

Scott and Anne had proposed that we look at this several weeks ago. They reported preliminary findings last meeting, and were going to check the non duplicate (in Ebsco) titles to see if they covered late 20th, early 21st century drama, and also if they were duplicated elsewhere. Scott reported that there were 31 unique titles, and that f those 15 had the potential to include late 20th early 21st century topics. These are not all FT. Scott took a look at Ebsco’s International Bibliography of Theatre and Dance, which is all full text and covers 130 books and 120 periodicals. Since we’ve determined that we do need to increase coverage - at least of indexing, and ideally of full text also - in both Theatre and Dance, we decided to go ahead and schedule trials of both of these simultaneously to facilitate comparing them. I did not put down the details in my notes, but I think that Laurie was set to arrange this, and did not need any more info from Anne or Scott. If that is wrong though - please add a comment.

Bibliography of Asian Studies

Scott looked into this, but found that it really did not add much to our JSTOR and Historical Abstracts coverage, and that more than that was not needed.

Subjects Needing more Indexing / FT Coverage

Sharon thought that she remembered that we were all supposed to be looking at our subjects and the ILL data, usage data etc. to determine if there were subjects that needed increases in Indexing / FT Coverage. No one else remembered this, and since she had not posted the notes from the previous meeting this made sense. However, some subjects were discussed.

Environmental Studies - Laurie brought this one up as an subject that we need more coverage in, but for which there is currently no good solution. The products that are in a reasonable price range for the amount that we would use them do not have the content that we need. The databases that have the content we need, are even larger than that and very very expensive. Laurie indicated that the product Earthscape that we had added a year or two ago was not really meeting the need. It was getting used, but she felt that people were not finding the content that they needed there, and so it was just misleading. ** She suggested that we drop this for next year, and everyone was agreeable to that.

Sharon asked Laurie if an increase in monograph spending, perhaps particularly edited volumes and/or adding some specific periodicals would help. Laurie answered that the monograph / edited volume idea had some real promise, and that we should perhaps look into that. She thought that it was possible that there might be a periodical or two that would help, but probably not. The main need is with the senior research students, and their topics are so varied that it is doubtful that one or two periodicals would do any good.

Religion - Laurie brought up that we’ve mentioned religion on and off in the past as a possible topic to increase coverage in. We discussed it a bit and the conclusion was that there was not enough happening here at the moment to justify an increase. We also noted that the addition of Historical Abstracts contributes quite a lot in this area.

No other subjects were brought up at this time.

Discussion on Usage Statistics

We went through Laurie’s Usage data and various comments were made :

• Medline - Sharon noted that this was probably down because more people were using the PubMed interface than the medline one.
• PsycInfo - Jeff had asked in a previous email about the possible decline in this that the graph shows - though we of course do not have complete 07-08 data yet. Laurie had been able to look at the data at the monthly level, and indicated that it is down about 1000 searches from where it was at the end of Winter term last year. Sharon noted that Spring Term definitely was the heaviest usage term, and so she did expect that much of the ground would be made up then. Laurie asked if the change in professors from Colleen to Kei might have made a difference. Sharon indicated that Kei was assigning the same kind of research projects that Colleen had been, but that she was not emphasizing PsycInfo itself as much (Colleen really pushed this)- so students probably are using Google more. The research methods class Psyc283 is this term, so that may make a difference.
• WWPS - noted how this is up already from the whole year of IPS - good news!
• ClasePeriodica - Sharon speculated that the jump in 06-07 was probably due to one student or faculty member using it for a project.
• LION vs MLA curve - in 06-07 we actually accessed MLA via LION - so that is why there was more LION than MLA. Then in 07-08 we got MLA separately from Chadwyck Healy, and so people are using LION less. People also seem to be slow to learn about MLA on its new platform - and it seems to still be accessible from LION - so there is some confusion. We don’t know if the MLA content that is still on LION is the full MLA content or just partial. Use of the two may also be down due to higher use of google and/or fluctuation of courses. We won’t really know for sure if there is a total downward trend till the Spring term data is added.
• RILM - we didn’t talk about this at the meeting, but I wanted to point out that the dip in this is probably due to Jeremy and Sarah’s Fall Sabbatical. They are the ones that use this in their courses. So, it will probably pick up in Spring, but still be down for the year potentially because of losing a term of use.

Database List Issues

A-Z List - Laurie proposed that we consider dropping ArticleFirst, Periodical Abstracts, Wilson Select Plus and ECO from this list. Not dropping access, and they would still be linked via SFX and could maybe keep in the general section of the subject list. This sparked a discussion of issues with the A-Z list. The basic dilemma is that the list is serving two purposes, and it cannot do both well.

1. As a fairly comprehensive A-Z list - where you look when you already know the name of the database, service or product you want to link to
2. As a selection tool - even though the Subject list is supposed to be the selection tool, because this is listed first (and perhaps because we more often refer to it in instruction and via ILL cancels because in those instances we are sending folks to a specific database), people click on it and use it as a selection tool.

Issues

• Because people frequently (perhaps even most often) use the A-Z list to decide on a database, the fact that it is long and lists things that we don’t really want to emphasize means it does not work well at this task.
• Even though people get to Periodical Abstracts and WilsonSelect Plus most frequently via SFX - there are times when people are told to go to it directly - for example in ILL cancels. Even though we can give them a link, sometimes they may still have trouble with that. Also these two databases don’t link that well to SFX, and so sometimes people need to go in direct if they cannot get to the Full Text via the SFX prompt. This means that there are a significant number of times when people need to be able to find a link to these databases from our webpage pretty simply. Going to an A-Z list is the most intuitive and direct way to do this.
• ArticleFirst is something that Sharon does direct students to in Physics and Chemistry classes. Although it is a very flawed database (no indexing, none of its own FT - though some links to PerAbs and Wilson), it still has significantly greater coverage of Chemistry and Physics journals than Ebsco’s Academic Search Premier. Usually Sharon provides a link to it in instructional materials, but again sometimes students are not going to have that info with them - so they need to have a way to find it easily from our webpage.
• ECO - we did not talk about this in the meeting - but this is not one that I think we want patrons to use at all - I thought we had it turned off - we can clarify if this is the case at the next meeting.

Possible Solutions - we discussed the following things, but we did not reach any conclusions and did not really feel that any particular solution so far solved the problems. It may be that we will have to prioritize the issues, and decide what is the best imperfect solution.

• Idea #1

1. Leave the A-Z list as a pure A-Z list (perhaps re-including some things on it - see discussion of non journal databases below)
2. Cull and Prioritize the Subject List - and add a star ranking system.
3. Place the Subject list before the A-Z one on the main page and in the drop down menu.

• Idea #2

1. Leave the A-Z list first, and cull it (perhaps also add star rankings) -
2. Keep the subject list second, also cull and rank it
3. Have a more complete A-Z list somewhere else - perhaps in the right column or under a site index or at the bottom and in Quick links

• Idea #3

1. Cull Wilson, Article first etc. from the A-Z list and perhaps add star rankings
2. Maybe or maybe not also get rid of “other Resources” column on the right.
3. Prioritize & Rank the Subject List, but include all the general databases (ArticleFirst etc.), and perhaps leave some/all of the “other Resources” here.

• Idea #4

1. Everything in Idea #3
2. Plus add everything to the library catalog so that a title search here would be a complete A-Z list of every kind of resource - books, journals, databases, packages, services, etc.

**Homework for Next Meeting**

Everyone will go through their subjects in the subject database list and rank the items as well as add or suggest deleting any.

Everyone will look at the list of general database and rank those and indicate what they want to leave out or add.

People will try to post this to the blog (note: I will make a general post, and then folks can add their rankings as comments).

• Non Journal Database Items and the A-Z list - a few months ago we agreed to separate out a number of titles from the regular A-Z list and to put them in a separate column to the right. Everyone agreed that this was the best thing to do at the time to help make the main list less unwieldy and to help eliminate confusion so that folks knew what the “Journal article” databases really were.
• However, Sharon expressed concern that this solution still makes it difficult for folks who have been told to use a specific resource or service, such as Choice Reviews, Credo, etc. to find those links. Most are not going to be able to make the distinction on their own that these will not be in the regular A-Z list, and so when they look there - they don’t find them. The column on the right is pretty prominent and easy to see, so that helps, but Sharon thinks it is still problematic. She speculated about whether or not we needed a complete A-Z list somewhere/somehow.
• Laurie pointed out that such a thing, if it were to include all electronic resources, even excluding periodicals, would just be huge and would be like trying to duplicate the library catalog. She suggested that shouldn’t we use and direct people to the library catalog as the main way to find all the resources.
• Sharon agreed that we definitely did not want to try to duplicate the library catalog in some list, just because things were online, and that we did definitely want to work more towards directing people to the library catalog to identify resources.
• She also indicated though that going through the library catalog was an extra step, and not all that intuitive for certain kinds of resources. That for some kinds of things people expected a more direct route. She speculated that there was perhaps a distinction that we could make that might differ slightly from person to person, but that we could not really adequately define between what needed to have a more direct link. For example - on the one hand it seems to Sharon that forcing people to use the library catalog to link to things like Choice Reviews or the Credo Package (not individual titles) seems problematic. That people just do not think of the library catalog as place to go to link to them. She wondered if some combined criteria - things that were “database like” along with things that were “large” (OED, Encyc Britannica) could be included on an A-Z list, while individual titles with only those very few exceptions would be linked just in the library catalog.
• Anne indicated that she thought that we could fairly easily come up with a short list of things that should be included along these lines, and then let the library catalog be the source for the rest.
• Laurie pointed out that if we put everything in the library catalog and directed people this way consistently that this should overcome most of those issues.
• Sharon conceded that this was true in some ways, but that she still felt somehow that having to use the library catalog to do search to link to what a patron might be thinking of in their mind as more of a service was problematic.

Misc Other Database Notes

• The CARLI list has not yet come out - Laurie thinks it should be any day now.
• Scott will be putting up the results of his analysis of the ILL data that Anne posted previously (note he has done this and it is in the Library/Reference folder).

Next Meeting

Because our time for deciding on the CARLI Databases is short, we will need to have a long meeting on Wednesday April 2nd.

**More Homework for Next Meeting**

If the CARLI list comes out, please review it in light of your subject areas, and come to the next meeting if possible (or if not the one after) with info on any of your subjects that you think we want to consider adding something for.

Caveat : I waited a long time to put these in, and of course now my written notes look cryptic - so these may be slightly off on some points. As always, if there is anything you think is wrong or left out - please add the info in a comment.

Ref Schedule issues

Reference over break - have a list of people on call. Look at the calendar to see who is here what days. Sharon will send out a draft list, and then we’ll pass it on to the circ staff.

Education Research Complete - Sharon had indicated in the agenda for this meeting that she thought that we needed to see pricing on this before we decided about it, and that she thought that if it was very expensive at all that because of ERIC’s comprehensive nature that we would not want it. However, Laurie pointed out that it might at substantial full text beyond Academic Search Premier. This would make a difference. I assume that we then indicated that someone would look into this - Anne? - but I have nothing in my notes.

Switching Databases to Ebsco Platform - We confirmed that we definitely want to switch the following :

• Philosophers Index
• ERIC

We are pretty sure that we also want to switch RILM and Art Abstracts, but Anne and Sharon still want to do one final double check with the faculty in those departments.

We confirmed that we do not want to move PsycInfo at this time.

CARLI Database Brokering List - Laurie will email us when this comes out.  She expects it around March 31st (note it did come out in early April, and Laurie has emailed everyone the link and password info).

Online Reference Collections

We noted that we did want to mostly wait till this summer to start reviewing this.   However, if there are some titles or collections on the CARLI list, then we can consider any that seem obvious choices or a deal etc.

In the long one when we look at this whole area - Laurie suggested that we approach this by thinking of specific titles that we want, and then seeing how those titles are available — rather than looking at collections as a starting point.  Everyone seemed to agree with this.

Database Plans

Revisions to the summary of the database planning procedure drafted in the previous notes :

• Laurie pointed out that we need a mechanism for dealing with things that come up outside of the planned schedule. There are times when there is a new database that we didn’t know about during the selection process or something is on sale, or a new gap in coverage arises. Everyone agreed that this was important. So, the conclusion was that if this happens - if it is in your subject area look into it initially to see if it is relevant or worth trialing - if so bring it to the rest of the group. If you want pricing before you decided, contact Laurie and ask her to explore that. If you see something that is really on sale and/or that you think we might be interested in (in an immediate way) that is not in your area, forward the info to the appropriate liaison. The main thing is, not to bring up or forward everything that comes along - but only things that for some reason warrant looking at before the regular place in the cycle that we have outlined.
• Laurie and Anne also brought up the situation where sometimes you want to do a trial connected to a specific course, and so again this would not necc. be during the time period designated for trials in the process outline. So, if this happens, feel free to ask Laurie to try to set up a trial that fits with your course. Again though, lets keep these to a minimum, and just use this when this kind of feedback is important to a decision about obtaining the product. We should not be doing trials of databases that were are not really contemplating getting at all.

Collection Development in General - Jeff brought up again that we needed to make plans to look at our collection development processes across the board, and not just for databases. He hopes that we can work towards “budget development” - where we are projecting needs, and then requesting changes to the budget to fit that - rather than just always reacting with the budget that we have.

Databases vs Periodicals

Jeff also brought up that we need to continue to work to define what we want to consider in the periodicals budget line and what we want in the databases line.

Sharon indicated that she and Laurie had been having some conversations on this, and that she (Sharon) was working on a spreadsheet that indicated for each major resources (not individual periodical titles that are obviously periodicals) the following 3 things :

• Which budget line is this resource paid out of
• Which librarian is responsible for the invoice for this resource
• Which librarian is responsible for managing the resource subscription.

She is still working on this, and hopes to have a draft that we can discuss soon.

Database Plan - Someone requested that Sharon put the draft plan for database renewal/selection as a word document linked to the blog in the sidebar, and not just leave it as a post. Sharon agreed to do this.

ArtStor

The question was brought up about what happens now about ArtStor. Jeff indicated that the next step is for he and Greg to talk.

Laurie indicated that if we were going to get this for 08-09, we’d want to know at the start of summer if possible so that we could get it on the agenda for the lab machine profile changes that happen in August. They would need to get the ArtStor software set up for that profile.

IIPA - International Index of the performing arts.

Scott reported that

• 240 journals were indexed here
• 60 of those were Full Text
• The coverage was from 1932 - Present
• There were 61 duplicate titles with Academic Search Premier

Scott will check for next time if there are duplicates with other databases, and will try to assess how much coverage there is of late 20th century and early 21st century that is not duplicated elsewhere. (Note, he did this, see the notes for the next meeting).

Scott mentioned that he would like to see a Google search link on the SFX menu, probably located at the end after the Library catalog and ILL links. What do you all think of this?

I got this message from OCLC and want to know what you think:

“You can have all FirstSearch comments submitted by your patrons sent directly to your library. Currently, if a patron enters comments in FirstSearch, that text is sent directly to OCLC Customer Support. When Customer Support receives the comments, they most often direct the patron to their library.

By adding the Reference Librarians’ email address in the WorldCat Services Administrative Module (known to some as the FirstSearch Administrative Module), the patron comments are sent directly to the library.”

There’s a service available called LibGuides and I think we should seriously considering subscribing to it. The cost is $899.00 per year plus an optional $60 per year domain hosting fee (so that we can have the knox.edu domain associated with it). This service allows librarians to easily create guides to resources based on subject or course or whatever. The service’s premise is that it’s based on web 2.0 features–it integrates with Facebook, allows for tag clouds, chat widgets, etc. I think the attractive thing for us is that you don’t have to have any skills or training in web design or html/css or anything to get a page up and running in a short period of time. I encourage you to visit the site and check out the real-life examples from libraries using LibGuides. There’s also a video demo on the site that you can look at.