One of the roles of the Information Discovery & Access Group (IDAG) is to stay current on technologies related to library searching. One relatively well-known search tool in the library landscape is VuFind, an open-source search interface built on top of Apache‘s Solr platform. We decided to try installing VuFind to accomplish several things:
- Gain experience and knowledge about this specific tool
- Find out more about search technologies in general
- Possibly perform some user testing to see how our patrons might respond to a different search interface
This is the first in a series of posts about the process of installing, configuring and testing VuFind here at Dartmouth. If you’re not interested in the technical aspects of installing the software, stay tuned for a subsequent post that will explore the interface itself.
Installing VuFind was not very difficult, but there were some obstacles. The first was platform. The installation instructions given by the VuFind Project focus on installing the software on specific server platforms: Windows, Ubuntu Linux, and Fedora Linux. The library’s servers run on Red Hat Linux, which is related to Fedora. However, in order to keep this experiment from impacting production services around the library, we decided to install VuFind using a Mac Mini as a server. For this reason, the installation instructions had to be treated as a rough roadmap rather than a set of commands that could be folllowed verbatim.
There are a number of components needed:
- The VuFind software, available on github
- The Apache web server
- A MySQL database
- The PHP programming language
- The Java Development Kit (JDK)
While most of the components are installed by default on a Mac, it was desirable to leave the system versions in place and install versions that matched what VuFind expected (and could be modified without altering the fundamental system components). Homebrew is a package management tool for the Mac operating system that makes this easy to do1.
Once all of these pieces were installed, and configured, we had a system that was operational, but no content in it.
The library’s catalog data is held in our Integrated Library System (ILS). The software we use is Sierra, which is provided by Innovative Interfaces, Inc. VuFind provides a search interface that complements the back-office workflow support needed to run library operations, which Sierra provides.
In the Sierra system, we used a feature called “Data Exchange” to create an export file of MARC records. The process of exporting approximately three million records took hours. The resulting file was 3.6 gigabytes of data. We copied the file over to the Mac Mini and imported the records. We had to make some adjustments to the configuration so that the record number from our Sierra system would become the unique identifier in VuFind.
VuFind indexes the records so that when you give it a search query it can find relevant data, which is why we import the records as a batch. However, there are some pieces of data that need to be updated continuously. The most obvious one is the item’s status – is it currently available or checked out? Since this data can change moment to moment, VuFind comes with plugins that can work with most of the major ILS systems, including Sierra. The plugin checks the current availability in real-time by querying the Sierra database at the moment when the item appears in a search result on VuFind.
Because VuFind is open source software, we are able to read and modify its code. Almost as soon as we got the system installed, we noticed a problem with the displays of certain records. We were able to trace the problem to a variable that wasn’t being initialized. We were able to edit the code to initialize the variable ourselves, and then submit that change back the VuFind community. This is one of the biggest advantages of open source software – work that we do to improve our own experience can benefit users of the software around the world.
Beyond changes to the code, there are many possibilities available to us in customizing the way the software indexes our data. There are specific notes fields in some records that are not searchable through our Sierra system without paying to have the data reindexed. With VuFind, we can configure the indexing to include whatever data we want to make searchable.
There are a number of steps that still need to be taken in order to have a fully functional search interface:
- Build some code to update the records in VuFind regularly. Currently the data in VuFind is a snapshot of what was in Sierra as of mid-October, but records are added, edited and deleted every day. We need to set up a pipeline so that the data in VuFind is kept up to date as changes continue to be made in Sierra.
- Filter out records that should be suppressed because the item is no longer held at Dartmouth.
- Integrate the library’s locations into the system. Currently the system doesn’t differentiate between the various branches around campus, but VuFind does support this – we just need to get the configuration done.
- Integrate patron data into the system. There are features of the software that require login (including tagging records, adding reviews, etc.). We don’t have VuFind integrated with the campus login system yet.
- Examine and adjust the indexing. The current setup is the generic one that comes included with VuFind, but to make it work best with our data will require the input of metadata experts in the library
- Get feedback from our users.
Look for another post soon which will get into some of the differences between the VuFind interface and the current search interface.
- brew install httpd22
- brew install php56 –with-homebrew-apxs –with-pear –with-postgresql
- brew install php56-intl
- brew install php56-mcrypt
- brew install mysql
- brew cask install java
- Install VuFind:
- wget http://downloads.sourceforge.net/vufind/vufind-2.5.tar.gz?use_mirror=osdn -O vufind-2.5.tar.gz
- tar xzvf vufind-2.5.tar.gz
- sudo mv vufind-2.5 /usr/local/vufind2
- cd /usr/local/vufind2
- php install.php