National Library of Australia
Kinetica Kinetica
This page is an archive page and will no longer be updated.

Return to Kinetica Homepage
Annual Users' Meeting 2000

Response time benchmarking:
Kinetica 3.2.1. and Kinetica 3.2.2

Introduction

Benchmark testing of Kinetica Client and KineticaWeb has been established to give an indication of the type of response times our customers can expect, and to assist in determining how various types of Internet connections contribute to response time.

Tests of Kinetica Client Version 3.2.1 and KineticaWeb were conducted in late May and early June. Testing of Kinetica Client was carried out again in August, following the loading of Patch 3.2.2. As the Patch was distributed in late July, not all libraries that participated in testing Version 3.2.1 were able to respond.

The range of response times for each search in minutes and seconds is set out below (least time to maximum time).

All results from participating libraries are recorded on spreadsheets. Copies will be distributed at the Annual Users Meeting. Please note:

  • library names/NUC symbols and ISP names have been removed to provide commercial confidentiality;
  • data on the Spreadsheets appears incomplete for some tests; this may be because this particular action is not carried out by the participating organisation; and
  • italicized data was either collected outside the recommended time frame (10.00 am to 3.00 pm), or the action taken was not part of the defined parameters. This data has been excluded from the analysis.

Kinetica Web

1. Twenty organisations provided benchmark data.

2. The number of test searches conducted by each library varied.
     1 minimum « 6 maximum

3. Summary of results

  • User name & password – welcome screen
         0:01 « 11:00
  • Search 1 icon – select 1
         0:01 « 2:10
  • Select NBD – build query
         0:01 « 0.30
  • Test 1: Title search and display
Step 1: search Step 2: display
0:01 « 1:47 0:01 « 0:47
  • Test 2: ISBN search
Step 1: search Step 2: view & display
0:01 « 0:28 0:01 « 1:25
  • Test 3: Title Keyword search
Step 1: search Step 2: display
0:02 « 0:148 0:045 « 0:40
  • Test 4: Name search
Step 1: search Step 2: display
0:02 « 0:44 0:056 « 0:33
  • Test 5: Title Keyword and Date search
Step 1: Search Step 2: display
0:03 « 2:12 0:04 « 1:50
Note: This test was lengthy as it involved two time consuming actions, combining a keyword search with a secondary index search. This combination of searching terms is discouraged in KineticaWeb training courses.
  • Test 6: Browse
Step 1: search Step 2: display
0:03 « 2:12 0:04 « 1:50
  • Test 7: Edit holdings
Step 1: find – extended view Step 2: edit holding Step 3: save
0:02 « 11:30 0:01 « 0:34 0:01 « 0:15
  • Test 8: Request Marc Record
Step 1: find –extended view Step 2: edit holding Step 3: RMR request queued
0:01 « 0:29 0:01 « 0:27 0:01 « 0:09
Libraries not wanting to download unwanted records or not using this facility did not carry out this test.
  • Test 9: Keeping record
Step 1: keep record Step 2: tick, click USMarc & save Step 3: choose where to save record and save
0:01 « 0:47 0:009 « 1.49 0:01 « 6:00

Kinetica Client: 3.2.1 and 3.2.2

1. Number of organisations providing benchmark data:

3.2.1 21
3.2.2 11

2. The number of test searches conducted by each library varied.

3.2.1 2 minimum « 11 maximum
3.2.2 4 minimum « 8 maximum

3. Summary of results

  • Record time until Main Menu appears
3.2.1 0:05 « 4:22
3.2.2 0:02 « 1:42
  • Click on Catalogue – Main Menu appears
3.2.1 0:02 « 5:00
3.2.2 0:02 « 0:21
  • Test 1: Title search and results screen
  Step 1: search Step 2: results screen
3.2.1 0:04 « 1:00 0:02 « 1:00
3.2.2 0:04 « 0:22 0:02 « 0:4617
  • Test 2: ISBN search
  Step 1: search & display
3.2.1 0:02 « 2:34
3.2.2 0:02 « 0:1938
  • Test 3: Title Keyword search
  Step 1: search, find summary Step 2: select, click, display
3.2.1 0:02 « 1:13 0:016 « 0:47
3.2.2 0:03 « 0.1344 0:02 « 0:10
  • Test 4: Name search
  Step 1: search, find, summary
3.2.1 0:02 « 2:22
3.2.2 0:02 « 0:3434
  • Test 5: Title Keyword and Date search
  Step 1: find, summary
3.2.1 0:03 « 2:22
3.2.2 0:09 « 1:28
Note: This test was lengthy as it involved two time consuming actions, combining a keyword search with a secondary index search. This combination of searching terms is discouraged in KineticaWeb training courses.
  • Test 6: Scan
  Step 1: search, scan, summary
3.2.1 0:02 « 15:00
3.2.2 0:01 « 0:33
  • Test 7: Edit holdings
  Step 1: select, screen appears Step 2: find record, edit screen appears Step 3: add holding, save line, save Step 4: OK, edit bibliographic item displayed
3.2.1 0:02 « 4:23 0:017 « 6:00 0:008 « 1:00 0:018 « 1:17
3.2.2 0:01 « 0:36 0:02 « 0.22 0:01 « 0:32 0:02 « 1:13
  • Test 8: Creating a record
  Click new bib item, worksheet, time to work 3 items
3.2.1 0:019 « 6:00
3.2.2 0:03 « 0:27
Libraries not wanting to download unwanted records or not using this facility did not carry out this test.
  • Test 8 (a): Title index
  Step 1: bib item worksheet – index display Step 2: edit/ heading/new time from index display to heading edit box Step 3: save heading, time from edit heading box to bib item worksheet
3.2.1 0:03 « 8:00 0:007 « 2:00 0:007 « 1:36
3.2.2 0:02 « 1:36 0:01 « 1:31 0:01 « 2:00
  • Test 8 (b): Name index
  Step 1: index lookup – time Bib item worksheet to index display Step 2: edit/ heading/new time from index display to heading edit box Step 3: save heading, time from edit heading box to bib item worksheet
3.2.1 0:036 « 1:00 0:006 « 1:00 0:01 « 5:00
3.2.2 0:02 « 0.24 0:01 « 0:07 0:01 « 0:49
  • Test 8 (c): Subject index
  Step 1: index lookup – time Bib item worksheet to index display Step 2: edit/ heading/new time from index display to heading edit box Step 3: save heading, time from edit heading box to bib item worksheet Step 4: click OK, record time until edited bib item displayed
3.2.1 0:03 « 3:14 0:007 « 1:00 0:01 « 5:00 0:02 « 34:00
3.2.2 0:02 « 1:04 0:01 « 0:50 0:01 « 0:13 0:02 « 1:35

 

Discussion

1. The effect of the 3.2.2 patch demonstrates improved response times for the small number of libraries able to participate in the testing at very short notice. Other libraries in the test group plan to provide their data in the near future.

2. The type and speed of the connection a library uses dictates the response time. For example, a 56 K modem connection will be extremely slow in comparison to a 2 Mb frame relay. One library reported:

"Our situation has changed since the last benchmarking test we conducted. Previously, our ISP was ... and our link to the Internet was via a 56K modem with one Client being used. The Client was loaded onto a stand alone and we recorded very slow response times. Now our IT Service Provider ... is endeavouring to run the Client on our LAN and they are testing it with our firewall, as yet this has not been perfected. We hope it does succeed as we are rather pleased with the response times we have recorded with the 3.2.2 version and our new situation compared with our previous setup using the 3.2.1 version of the Client on a stand alone."

This outcome demonstrates the benefits of:

  • using a major ISP providing greater capacity; and
  • moving from a modem to a more robust connection.

3. Choosing the right ISP is crucial; it is important to select a ISP set up to support commercial services. Such a provider is more likely to have a wide bandwidth capable of handling large traffic flows and the ability to respond quickly to increased traffic demands, than a provider who is supplying the domestic market on what is likely to be a smaller bandwidth with less flexibility.

4. Configuration of local networks can also have a bearing on response time due to information flows around the LAN/WAN. Security measures (eg firewalls) and the level of traffic (both incoming and outgoing) can also affect Internet response times. Trace-route data provided by a university library, demonstrated this point; once the data entered the university’s LAN it made numerous "stops" and was timed out before reaching its destination.


Draft agenda Kinetica Annual Users' Meeting 2000