Project ARS: Story

This is a mini-diary of the process from where we moved from an old building into a new one and included a new ARS in the process.
(More below)

Our vendor for the ARS portion of the new library was HK Systems.

Our vendor for the interface used by the public to find books stored in the ARS was done by Innovative. (This includes the web based system that allows you to check the status of items, location, availability, and make requests for items from the ARS.)

The story:

I cannot offer a story for the original planning of the new Jean and Charles Schulz Information and Technology Center when it was originally being drafted. The original specifications for the new building were created long before I arrived to work at the SSU Library.

After working here for a few years, I was informed of the new building project. The most exciting part of the project to me was the inclusion of the ARS for automated retrieval of books and items in the library collection. Susan Hagius (the library director at the time) assigned Sandra Walton (Special Collections Librarian) to work on many of the new building issues. Sandra Walton worked many long hours to keep track of all of the new building planning, and organize everything. She mentioned to Cheryl Esch (Library System Department) that she had some technical questions about the new building and the ARS. Cheryl Esch suggested that Sandra Walton talk to me to see if I could examine the technical information on the new building. This is where I came into the project.

It all started easily enough with Sandra Walton and I asking Bruce Walker (Jean and Charles Schulz Information Center Project Coordinator / Director) if we could see the building RFPs (Request For Proposal) for the Power And Signal, Networking, ARS, and ETV (Electronic Track Vehicle). Bruce Walker arranged for me to pick up electronic copies of these, and I read through them. After careful review, I realized the ARS needed the most attention from us in the library.

CSU Northridge has an ARS in operation. Their ARS has been in use for a long time. I made arrangements to go visit the CSU Northridge Library and visit Eric Willis (CSU Northridge Library ARS and Systems member) as he was the system administrator of their ARS Manager/Server. During the visit with Eric Willis, we discussed improvements to the ARS system. He gave me contact information for their ARS vendor, which had changed names since CSU Northridge contracted to have their ARS built.

The visit with Eric Willis was enlightening. I was able to see an ARS in operation, and get some suggestions for our own ARS. Sandra and I spoke about this, and she suggested I start modification to the SSU RFP on our ARS. I proceeded to include a few pages of additions based on the input and suggestions from Eric Willis.

For several months, I pondered upon the use of the ARS. I tried to find problems, issues, and conflicts then solve these. I found many places to improve the RFP for our operation and update it to make it more modern. The original 35 page RFP had grown to well over 70 pages when I finished with all of my inclusions.

I proceeded to contact HK Systems to see what kind of operating system they could offer us for their servers. Northridge was using a UNIX system for their operation. Eric Willis had many good things to say on the stability of their system. It has been running for about a decade or longer at CSU Northridge. We wished to also have stability in our system. The early conversations I had with HK Systems' associates gave us little hope for a UNIX system. Everyone I contacted at HK Systems told me they no longer offered a UNIX system, and only worked with MS Windows NT. My experience at the time with MS Windows NT 4.0 and SP2 left me feeling that Windows NT 4.0 was not an ideal platform for running a critical service that needed to be in operation with tolerable uptime. I knew of many UNIX boxes that were left running for double digit months, and even years at a time. With comparing Windows NT - a new technology, and UNIX - a stable, tried, and tested technology, we favored a UNIX system for our operation.

After about two weeks, my continued attempts to locate someone at HK Systems willing to talk about UNIX servers paid off. Mike Marlowe (presently Mike_Marlowe@Daifukuamerica.com, but before we had decided on a vendor he was Senior Account Manager at HK Systems, now working for a different company called Swiss Log) called me up and let me know they have, sell, and continue to support many UNIX, and UNIX-like operating systems. He was contacted by one of the people I contacted, and followed up by contacting me. He assured me that HK Systems still offered UNIX solutions, and had no plans on discontinuing support for their UNIX servers.

Then I took a trip to Salt Lake City, UT to visit their Salt Lake City offices, and also their production plant in Bountiful, UT. I spoke with many of their people (Don Davis (Assistant Plant Manager), Michael S. Johnson (Vice President - Plant manager), and I think Roger Uhlich (Principal Engineer) and Tracy Larson (Principal Engineer-Software)) to take a tour of their work-place in downtown Salt Lake City, Utah. While at the Salt Lake City offices, they introduced me to one of the Engineers who told me they were willing to work with Linux. He said they started to work with Linux on one other projects, but later had to change away from Linux as their Operating System when their customer changed their mind. (To the best of my knowledge, we have the first ARS to use Linux as the core for the Manager/Server.) It was during this visit that HK Systems informed me that the UNLV had a new library that was going to have an ARS made by HK Systems.

I was impressed by the professionalism, and manners of the employees I met while on tour of the Bountiful, Utah production plant and the downtown Salt Lake City, Utah offices. When I was able to get some time away from the rest of the group mentioned above, I was able to speak with the people in the production plant that actually make the equipment and parts. Even without the pressure of the managers with whom I had been touring, the employees had good morale, and good things to say about HK Systems. (From my experience, a company that treats its employees with respect find employees that work hard and also respect the company.) I heard no dissension from any of the employees at HK Systems. From the newly hired people, to the veterans, all spoke highly of their company.

Next, I scheduled a visit to UNLV to visit their library systems department. I spoke to them about their planned ARS, and asked them how they viewed HK Systems. They offered examples of how HK Systems had done very well to meet their needs and in some cases go farther in service and support for the system than was expected.

The reviews from Northridge, UNLV, and the professionalism of everyone at HK Systems starting with Mike Marlowe (now working for http://www.daifukuamerica.com/ but at that time working for http://www.hksystems.com/) through to the people I met on my visit to Utah was very good. I thought that this seemed like a good company to have do work for us with our ARS. They have serviced other library locations, they had good reviews from the systems employees at these locations, and they seemed to me to be intelligent, thoughtful, hard working people.

I completed the work on the RFP for the ARS, and we submitted it through to Bruce Walker and the campus. From there, it was out of our hands while it was put out for public review and open for any company to offer a bid.

Mike Marlowe (now working for http://www.daifukuamerica.com/ but at that time working for http://www.hksystems.com/) contacted me and we worked out a few items in the RFP where a contention for service, or conflict was created that needed resolution. After these items were resolved, we waited a bit longer. (For example, I requested a Linux system on an Intel platform with 64-bit CPU support in the RFP. I originally requested this in anticipation that the Intel Merced would be available. Not only was this release delayed by Intel, but our building project was coming through on time! This caused our RFP to request a product that did not yet exist.)

HK Systems returned a counter to our RFP, and though it did not have all of the features desired in the original RFP, the most useful ones were available. A number of features would have been nice additions for some applications, but the added complexity could have led to conditions for conflict for the employees tending to the system on a day-to-day basis. The items missing in their response were dropped as a result of our conversations and clarifications on directions and responsibilities for the ARS. We agreed that simplicity decreases risk for errors and makes learning cycles for training of employees shorter. These were excellent selling points in weighing added functionality with complexity of use.

I was happy to find that HK Systems won the bid. Later I was contacted by Floyd Roach (HK Systems Project Manager), David B. Simpson (HK Systems Systems Engineer), Yen Cheng Kang (HK Systems Principal Engineer-Software) , and again by Tracy Larson (HK Systems Principal Engineer-Software). We worked out our details, and tried to hammer out some semantics on ordering and processing.

Marilyn Kamp (Library Circulation Administrator and Manager) and I then took a visit to Salt Lake City to review some final interface issues, and features in the system. We discussed project guidelines, system logic, user interface, and functionality. We started to see the project take form and were excited to see the emerging product.

Over many more months of e-mail discussions, we further ironed out details on the system. Eventually, SSU was invited to visit Salt Lake City for a review of a beta system of their ARS Manager software. Marilyn Kamp, Greg Tichava (Library Pre-shelving and Circulation), and I went to this review. We brought a video camera borrowed from the campus media services and filmed our review of the system. (They were very helpful in providing us with a camera and extra battery for filming our review.) Things went well with our review, and I was excited to see the interface up and running. We found a few items that needed to be repaired, and some user-interface issues with meaning of text not conveying what could be easily understood by non-techies. Beyond these small items, we could see most of the software side of the project was ready!

More time passed and I continued to work with David Simpson, and Yen Cheng Kang to deal with other minor issues on user interface.

Andru Luvisi (Library Programmer/Analyst), Phil Huang (Library Systems Department Head), Inna Moroz (Student Assistant) and I borrowed Cheryl Esch's Truck and delivered 3 tables for temporary use with the ARS Manager Servers, printer, and eventual location of the III Server. During these many visits where we brought over tables and chairs, I was introduced to Rick Stallworth (HK Systems Hardware Engineer.) On one of these many trips, we picked up one of the ARS Manager Servers for temporary re-location to our old library building for Host Interface testing with the III Server.

David Simpson and Yen Cheng Kang came to visit us to test the Host Interface with the III server. Our III Contact was Beth Hoffman, and the Host interface test went well. We tested the ability for both machines to transmit and receive a variety of different messages. After this test we were certain that both machines could communicate to each other enough information to offer a good foundation for future performance of the system. All that was left was for III to integrate their module used in testing into their service, and handle the Host Interface messages, and for HK systems to properly handle their messages from the Host interface.

Work for the ARS side of handling messages is completed, and has been tested. Development with III is on-going and will continue to develop over time. We expect the III system to have enough of the Host Interface conversations to offer patrons the ability to perform pick requests from the ARS. Minor details on updating the status of each item within the III System to notify the patron when a book has been requested for removal from the ARS by another patron near the same time, or another patron has picked a book from the ARS for in-house viewing, should be resolved at a later date. We will review desired additions to the III System and prioritize them based on want, need, and necessity.

We delivered the ARS Manager back to the new building, and later brought over the III Server for a live Host Interface test. Later we received training for the system, and started the system acceptance for the university. System acceptance included many kinds of tests including test loads of some library items, and bin dividers. About 1000 low-use items were stored in the ARS in this test. In parallel with our system acceptance, Terry Cheney (SSU Facilities Services) worked with Rick Stallworth to better understand the ARS.

On Tuesday, June 6, 2000 at 4:00pm, we started a meeting with a number of people related to the ARS project. In attendance were Floyd Roach, David Simpson, Rick Stallworth, Bruce Walker, Marilyn Kamp, Greg Tichava and me. After the formality of a few last signatures, at 4:45 PM, the university accepted ownership of the ARS.

Though we did perform a test of the ARS side of the Host Interface to the Snoopy system, we need to verify the functionality of the Snoopy side of the connection. This may be a source of ambiguity for the non-technical people, but I will try to explain this:

When we tested the host interface with III, the ARS was using a completely integrated communications systems. However, the III system was still being configured and tested. While the Snoopy (III) system was not ready, III decided to pull out the module in their server that communicated with the ARS. This special module could then be tested with the ARS to make sure they would both communicate as expected. A few small problems were found on both sides and fixed. At the completion of the first host interface testing, we saw the expected conversations take place between both side without error.

Now we need to arrange for a test with the Host Interface Module re-integrated into the Snoopy (III / ILS) system. This will test to make sure the rest of the Snoopy software reacts internally as expected to the communications received from its Host Interface Modules through the physical Host Interface. We do not expect any problems with this, but it should still be tested.

After the second (and expected final) host interface test, we will then look to methods for transfers of records to the ARS. First we will test an advanced system for automated record transfer on demand from the Snoopy system to the ARS Manager. If it is found that this system is not ready before the end of the first week in July, we will look to manually insert a nearly complete database of library items into the ARS database. Either of the above methods will be necessary for the initial physical loading of items into the ARS starting sometime after July 31, 2000. The database transfer of records to the ARS must happen before we can allow the physical loading of bins because the ARS needs to know about the items being stored in bins.

June 13,2000 we met with William G. Overton from William B. Meyer Incorporated to discuss ARS related moving issues. This company will be dealing with the migration of computers, furniture, and most importantly the books to the new building. This company has openly accepted the challenge of loading the ARS with books and library items. Time-lines are still being worked out for this move.

For the week of July 17-21, 2000 Rick Stallworth (HK Systems) came out for a visit to exercise the cranes and hardware of the system.

On July 20, at 9:15AM PDT the host interface between the III Server and the ARS Manager was brought on-line. Over the whole day I worked with Sandy Heft (Library Technical Services), Beth Hoffman (III) and Yen Cheng Kang (HK Systems) to make sure that we could perform "Item Additions" from the III database to the ARS Manager database. Though the processing is a bit slow, we expect to be able to prepare the ARS database for when the movers will need to load the ARS with the physical items.

After the basic host interface testing was completed, we started the database load of the ARS manager. This is expected to take up to 6 days.

July 24, 2000: Minor problems have slowed our load of the records into the ARS over the host interface, but we feel it should still be possible to finish the electronic load of records into the ARS Manager of items expected to be moved to the ARS before the items are to be moved into the ARS. We continue to work with III to find solutions, and add features for patron interface.

August 2,2000 @ 9:00am : All but a few thousand records have been transmitted to the ARS Manager over the active host interface.

August 2,2000 @ 11:30am : Today we discovered that III has included support for the demand based load-item for the move! This has helped us a great deal to store items that would have otherwise been left out until after the move. We were happy to see III work so hard, and so fast to deliver this feature to us. The inclusion of this just in time for our move allows us to progress forward with the move of books into the system quickly.

August 2,2000 @6:00pm : Today we finished the first full day physical load of items into the ARS. All 6 End-Of-Aisle-Workstation (EAWS) were in operation, and all three cranes were in use over the full day. Here is a table showing the load progress and how fast we were able to load items into the ARS for the initial load.

(The above table can also be seen on a separate page for easier printing.)

August 9,2000 @ 11:52AM : All of the cranes stopped accepting commands. After about 1 hour of research and trouble-shooting, I determined it was a problem with a piece of solid state networking equipment called a "terminal server". I called HK Systems, and spoke with Kang about this, and he relayed the method for the terminal server to be reset and reprogrammed as well as the location of a serial cable to direct communications with the terminal server. I attempted to reprogram the terminal server, and it would spontaneously reboot itself and forget the programmed settings! After 3 attempts of this, and continued examination of the box, the 4th attempt caused the terminal server to retain its settings. We believe this component may have an intermittent short. HK sent us a replacement via overnight delivery.

August 10,2000 @ 3:00PM : The terminal server did not arrive at the boiler plant for Terry to relay it to us, but we know HK shipped it as we have the tracking number.

August 10,2000 @ 6:00pm : A tracking of the package with Airborne's web page for tracking shows it was delivered to Santa Rosa at 11:45AM on August 10th. I left a message with John Griffith (SSU Receiving) asking about the package.

August 11,2000 @ pre 8:00am : John Griffith left me a message on my voice mail about the package. I contacted him at the number he left on my voice-mail, and he arranged for a student to deliver the package right away. (Thanks John!) I have scheduled to replace this part at 5:00pm so I have a 16 hour period to deal with any other problems that might come up as a result of replacing the previous terminal server. ( I believe I could make this change in about 15 minutes or less, but prefer to have more time in case of unexpected events.)

August 11,2000 @ 8:00am : The crane on Aisle 3 had a problem while returning a bin. A proximity sensor was damaged preventing the bin from being returned. We have a call in to have this part sent to us fast. Production loading of the ARS will be down by 33% while one of the cranes is down.

August 11,2000 @ 11:00am : I have confirmation from Jean (HK AS/RS Parts) that 2 of these parts will be shipped to us via Fed Ex overnight with Saturday Delivery. Public Safety will act as receiver of this package as local receiving may not be open on Saturday. We hope to get this sensor by or before 10:00am so we may continue our processing on aisle 3.

August 11,2000 @ 5:15pm : I replaced the terminal server with the new one. The change-over and reprogramming went flawlessly and took about 8 minutes. So far this new one works very well. (The old one continued to function without failure for the rest of the first day of problems and all day August 11th. I replaced it because we did not want to take a risk that the problems we experienced could come back again.)

August 12,2000 @ 8:40am : Maintenance was conducted on the cranes/aisles and the tick locations for the two bins blocking aisle 2 production were updated. Aisle 1 and 2 are operational. The part for Aisle 3 was loaded onto a Fed Ex van in Petaluma at 8:20am.

August 12,2000 @ 10:00am : Fed Ex delivered the parts we needed on Aisle 3 to the right place on campus! (We did not want to try Airborne again after the mistake from before.) Terry replaced the part, calibrated the machine, tested the machine, and made it ready after a member of Public Safety went out of their way to help us out and drop off the part to us. Aisle 3 was up and running by 11:00am! All three aisles are up and running again! (We now have an extra one of these finger sensors in stock in case of a future mechanical failure.)

August 14,2000 @ 3:45pm : A loaded bin would not return to the rack space. This does not affect patron pick requests but did force us to temporarily hold it off to the side at a workstation and shut down that workstation until the technician (Terry) can adjust the bin return coordinates. This has happened rarely in our daily processing of hundreds of bins and was expected to come up occasionally as we start to intensely use the system for the first time.

August 16,2000 @ 3:00pm : We have started working with III to bring up the patron access to the items stored in the ARS. Presently, a few items show up in the web based III sessions, but III needs to make a few more changes for the public side of things to work. Minor problems with the Host Interface today created problems for the movers.

August 17,2000 @ 10:00am : More testing of Patron Based requests is being performed today. We have had very good progress, and have isolated a problem for the first half of the patron request. III is working on a fix for allowing the public to request items from the ARS. (We respectfully ask people to not make requests via the web until after we know it all works. We are trying to meet the deadline of August 23, 2000 before 8:00am.)

August 18,2000 @ 12:30pm : More host interface testing took place today. III Upgraded their Host Interface services, and this has solved the patron status check problem from yesterday.

August 18,2000 @ 3:45pm : Marilyn Kamp, Francia Phillips, Raye Lynn Thomas, Tim Huston, and Sandy Heft worked with me to send through many requests for books stored in the ARS. Over 150 items were processed through to the ARS over the host interface with ZERO host interface errors. We are trying to simulate conditions of a live library setting with many users making requests. We need to code up some web pages in HTML so the errors reported to the patron are more descriptive to the status of the items they are looking to examine.

August 19,2000 @ 12:00pm : We started the final load of books into the ARS for the move.

August 19,2000 @ 6:00pm : We think all of the items expected for move into the ARS have now been transferred.

August 20,2000 @ 5:00pm : I have provided a copy of all items scanned for ARS locations and items not scanned for ARS location to Sandy Heft. She will look to synchronize the III SNOOPY database so the review of items by patrons will properly show ARS stored items as actually being stored in the ARS, and items not in the ARS as not stored in the ARS.

August 21,2000 @ 11:00am : I finished performing the picks and stores on the 150+ items requested by library employees for pulling from the ARS.

August 21,2000 @ 1:00pm : I found, fixed, and tested a problem with the ETV Program service on the ETV Server. Now the ETV seem to work as demonstrated to us. (ETV= Electronic Track Vehicle. These are not part of the ARS, as they were made by a different company called TransLogic, but are part of the process for getting books from the ARS to one of 3 library locations - 2 of which may be used for public pickup: 1st floor circulation, and 2nd floor circulation.)

August 22,2000 @ 1:25pm : I found and fixed the problem with printing! The ARS side should be ready for business Wednesday morning.

August 22,2000 @ 3:30pm : I processed a few more pick requests from the III side, and it still seems to work. The ETV and ARS seem to work.

August 28,2000 @ 8:00am : News paper article in the press democrat discusses the ARS. A link offered in my links page.

To be done:

• Synchronization of III and ARS databases: Sandy Heft and III need to synchronize the data I offered them with the Snoopy or III catalog and database. Once this is done, the items physically stored in the ARS will show up as stored in the ARS while items not stored in the ARS will show up with another location. (To my knowledge, this is being processed on the afternoon and evening of the Tuesday before we open! Go Go GO!)
• III may need to add a function to limit processing of pick requests during off hours if that is abused.

  Please note: We will need to work with III to improve patron access to the ARS. We (the university or library) cannot effect these changes on our own, but are certain that performance and function will improve over time as we are permitted to work with III. Assuming we meet our deadline, we will be going live with a system that has not been fully tested. We expect users to find bugs, and as they are found we will address them.

Thanks for reading!