Aimed solution for implementing the TEL Portal (draft - The

By Kevin Baker,2014-07-11 16:21
6 views 0
Aimed solution for implementing the TEL Portal (draft - The ...

IST 2000 25347/Aimed solution for the TEL test portal

     The European Library TEL

     IST 2000 25347

     Aimed Solution for

     Implementing the TEL Portal

Name of Client:

Distribution List:

    Author: Theo van Veen, Annette Siegenthaler, Bill Oldroyd

Authorized by:

Contractual Date:

     0.4 (draft version) Date: 13.8.2003 Issue:

     WP4 Interoperability Testbeds Workpackage:

Deliverable Number / Type / Nature


Total Number of Pages:

Contact Details for Organisation:


    IST 2000 25347/ Aimed solution for the TEL test portal

    Table of Contents:

1 Document Control 1

    1.1 Abstract 1

    1.2 Keywords 1

    2 Management Overview 2

    2.1 Executive Summary 2

    2.2 Scope Statement 2

    3 Introduction 3

    3.1 Glossary 3

    4 Development of the TEL portal architecture. 4

    4.1 The conventional portal architecture. 4 4.2 The SRU test interface architecture. 4 4.3 The introduction of the SRU Z39.50 gateway. 5 4.4 The introduction of xslt middleware. 6 5 Concepts incorporated into the SRU test portal 7

    5.1 Content and layout/functionality are separated 7 5.2 Requests are SRU compliant 7

    5.3 Multiple requests are launched and processed in the browser 7

    6 Benefits, drawbacks and required developments 9

    6.1 Benefits 9

    6.2 Drawbacks 9

    6.3 Required developments 9

    7 description of the current TEL test portal 11


    IST 2000 25347/Aimed solution for the TEL test portal 1 DOCUMENT CONTROL

    Issue Date of issue Comments

    0.1 04 August 2003 First Draft, request for comments until

    0.2 12 August 2003 Second draft with comments from BO and TVV

    0.3 13 August 2003 Third draft with some minor corrections

    0.4 14 August 2003 Forth draft with comments of JK

1.1 Abstract

    This document describes the solution of the current SRU test portal and outlines the amendments that should be 1applied in order to implement the operational TELService system (including a TELService interface), which 1would be developed after completion of the TELProject . The aimed concept declines from the original idea of

    a central commercial portal and it is described how this change in thinking is achieved. Additionally the advantages and drawbacks are described and developments that are needed to limit the impact of these drawbacks.

1.2 Keywords

    TEL, National Libraries, Search and Retrieval on the Web, Z39.50 protocol, ZiNG, SRU, XML representation, Distributed Bibliographic Resources, Digital Publications

     1 For clarity the following terms are used in this document:

    TELProject refers to the present project which is an EU subsidised accompanying measure

    TELService refers to the operational service for which the present project as an accompanying measure is preparing and to which the partners have committed. TELService is a shared central service that gives access to local services offered by partner libraries.


    IST 2000 25347/ Aimed solution for the TEL test portal 2 MANAGEMENT OVERVIEW

    2.1 Executive Summary

    This document is supposed to serve as a basis for formal assessment of the desired solution for the TELService.

    It should as well serve as basis for the technical specification that has to precede the implementation of the

    TELService. It won’t become a formal Deliverable, but remains as an internal paper to the project. 2.2 Scope Statement

    This document describes the concept of a portal running as a part of the browser in relation to a conventional

    portal running as central service and some required developments are described for turning the original test

    interface on in which this concept is used, into am initial production system. It will not replace a detailed

    technical description of the complete and final TELservce.


IST 2000 25347/Aimed solution for the TEL test portal


The TELservice was original thought to be a commercial portal giving access to distributed metadata via Z39.50

    and to a central index via http/XML. At the start of the TELproject the Z39.50 Implementers Group started the

    specification of a new standard for search and retrieve (SRU/SRW) as a part of a broader initiative called Z39.50

    internation Next Generation (see SRU being Search and Retrive via URLs was adopted by TEL for http/XML access of the central index. For testing the implementation of this

    protocol a test interface was developed using the XSL capabilities of the Internet Explorer. This test interface

    (see was based on Javascript and XSL and showed to be a simple but

    powerful tool to demonstrate potential portal functionality, mainly because it was easy to integrate search and

    retrieval, navigation and presentation just by combining Javascript with XSL. This interface was also used for

    the metadata development of the TEL project.

    When it showed to be easy to build a gateway to Z39.50 systems, it was expected that this solution might be a

    low-cost and powerful alternative to a commercial portal. As such it was advised to the TEL steering committee

    as an extra option.

3.1 Glossary

    CQL Common Query Language. CQL is the query language for SRW and SRU, and may be used by

    other protocols as well. CQL is designed to be human readable and writable, while maintaining

    the expressiveness of more complex languages. SRU Search and Retrieve via http. A protocol using http/XML still under development and being

    used for the TEL http/XML testbed described in this document. URL Uniform Resource Locator. A string that identifies a resource (document or service) in the

    web. It can serve as a web address for a hypertext link. XML Extensible Mark-up Language. Agreed record syntax in TEL. XSL XSL is a family of recommendations for defining XML document transformation and

    presentation (Internet: XSLT Part of XSL. An XSLT stylesheet specifies the presentation of a class of XML documents by

    describing how an instance of the class is transformed into an XML document.

    Z39.50 International standard (ISO 23950) that defines a protocol for computer-to-computer

    information retrieval.


    IST 2000 25347/ Aimed solution for the TEL test portal


In this section the aimed solution for the TELService is described by the development of the ideas of the TEL

    architecture from the original ideas via implemented test interfaces to a production architecture that still has to be


4.1 The primary portal architecture.

The primary portal solution aimed for the operational TELService is shown below. It was a conventional

    approach: A central portal receives an http request from a browser and converts this request to Z39.50 requests,

    broadcasts these requests to different Z39.50 targets simultaneously, waits for the responses and converts the

    results into an HTML presentation. In case of TEL the request is also converted to a SRU request and sent to one

    or more SRU servers, including the central database of harvested metadata.

    Figure 1. The primary portal architecture

The SRU protocol did not yet exist at the start of the TEL project it was originally specified as a native http/XML

    protocol. Of course at that time there were no commercial portals available that supported http/XML or SRU and

    for testing purposes it was decided to separate the Z39.50 testbed from the http/XML testbed.

4.2 The SRU test interface architecture.

As there were no SRU-clients or SRU-portals available at the beginning of the project a test interface needed to

    be developed. This SRU test interface was based on the capability of Internet Explorer to process XML with

    XSL stylesheets and the architecture used for testing is shown below. The actual portal is written in Javascript

    that is contained in an XSL-stylesheet.

    Both the list of targets the user may choose between and the returned responses are XML documents, which are

    transformed by using XSL. The search requests are broadcasted simultaneously to the list of SRU-targets via

    Javascript just as conventional http GET-requests that are sent by the client’s browser. The responses are XML documents as well and are transformed to HTML for presentation to the user. Again this is done in the client’s

    browser. In other words: the browser is accessing the SRU targets directly without any server component in

    between. This has a number of advantages, like performance, the ability to show results without waiting for the

    last target to respond, the possibility for user to specify his own stylesheet for a personalised presentation, etc.


IST 2000 25347/Aimed solution for the TEL test portal

    Figure 2. Architecture for the SRU test portal.

4.3 The introduction of the SRU Z39.50 gateway.

In order to provide access to Z39.50 targets, a central Z39.50 to SRU gateway has been developed. However, it is

    expected that Z39.50 data providers will either provide native SRU interfaces for their targets or make their data

    accessible via their own local Z39.50/SRU gateways in the future. The reason for this recommendation is, that

    most Z39.50 targets don’t behave in a coherent manner. This makes the implementation of a generic solution

    working for all targets a difficult effort.

The recommended architecture is shown below. This architecture is currently available with specific gateways

    for specific Z39.50 targets and with a central generic gateway. The gateways run as separate cgi-scripts on a

    single server. The need for specific scripts in addition to a generic script has to do with the non-standard

    behaviour of the Z39.50 targets. A generic cgi-script should be sufficient for targets with well-formed Z39.50

    behaviour and XML responses in conformance with the TEL guidelines for the record syntax.

    Figure 3. Architecture with Z39.50-SRU gateway


    IST 2000 25347/ Aimed solution for the TEL test portal

The gateway is currently implemented as a python script running on a PC under windows. The SRU protocol is

    implemented with limited support for CQL, which is the SRU query language. The query is converted to a

    Z39.50 query and the XML response is put in an SRU envelope. There is not yet support for conversion from

    MARC to XML in compliance to the TEL application profile. This will need to be developed by applying an

    existing tool for converting MARC to MARCXML and by applying XSLT transformations for conversion from

    MARCXML to XML according to TEL AP.

4.4 The introduction of xslt middleware.

The main drawback of the SRU test portal architecture is the dependency on a specific browser and specific

    security settings in the browser. So there are requirements on the client side system architecture that heighten the

    access barrier. Making the portal available for more browsers that support XML can reduce this disadvantage,

    but still there always will be some browser dependency. To overcome this a server side central service should be

    developed that can do the transformation from XSL and XML to HTML.

    In the next figure an extra middleware component is added to take over the xslt transformation from the browser

    and returning “browser-independent” HTML. This middleware component will replace the browser-portal but

    the browser-portal will be kept available at least for some time to base final decisions on the archtecture on the

    experience obtained with both archtectures. This middleware component is not yet available and needs to be 2developed, but there are free software modules available that allow a inexpensive and fast way of realizing this.

    Figure 4. Aimed architecture for TEL portal

This architecture is the aimed architecture for the TELService. The TELService Interface generates HTML

    pages from the SRU responses by means of XSLT stylesheets on a central server. However, due to the openness

    of the architecture it will always be possible to skip the central service component and provide the same

    functionality or extra functionality by means of stylesheets and transformation within the browser. It is not yet

    clear how this will effect future developments but we cannot neglect the fact that it will become very easy for

    others to provide additional functionality in this way.

     2 e.g. Apache’s Cocoon2.0. Internet: (For an short introduction to cocoon we recommend the article


    IST 2000 25347/Aimed solution for the TEL test portal 5 CONCEPTS INCORPORATED INTO THE SRU TEST PORTAL

    A number of concepts are incorporated in the current SRU test portal. These concepts have been applied by experimentation. They haven’t been subject to theoretical analysis in this project, but are described and

    discussed in the literature dealing with the implementation of web applications. Three major concepts can be distinguished that will be described below.

    5.1 Content and layout/functionality are separated XML pages generated by SRU searches are providing the content. The layout description and common functionality (e.g. translation of terms, generating new searches and generating links are stored in XSL(T) pages. Both types of pages are loaded by the client program via the http protocol. The content of an XML page is converted to XHTML by the commands in the XSLT script. The user views the XHTML page in the browser window.

    Keeping content and layout plus functionality information separated allows for consistent changes of the interface without major efforts. It also allows different interfaces to be provided, for example interfaces in different languages, by switching the XSLT page that is used to generate the XHTML.

    The XHTML version of an SRU response page could also be generated by the server. Of course, it is an advantage, if no server component is needed to merge content and layout/functionality, as this would allow easy web publishing by staff unfamiliar with server side scripting or other programming expertise. Unfortunately some older, widely used browsers do not support XSLT processing. So this solution depends on the use of modern versions of specific types of browsers (i.e. Internet Explorer, Mozilla, recent versions of Netscape, Safari). Another disadvantage is the fact that lax security settings are needed to allow the XSLT page to be hosted by a different domain than the SRU service page.

    5.2 Requests are SRU compliant

    Any request transmitted to a server for information retrieval uses the SRU protocol. So any client, which “understands” http, can be used to transmit an SRU request. Any version of any browser can be used without restrictions regarding security settings or other settings. Neither plug-ins nor other add-ons are needed. The results returned by the requested services are in XML. This makes it easy to process (for example, adding functions, sorting, de-duplicating) and to produce a “nice” output. The SRU protocol is less complex than the

    Z39.50 protocol making it easier to understand and implement.

    The downside of using only SRU is the fact that not all data providers support it. A provider that does not support SRU needs a gateway function to transform the SRU to the Z39.50 protocol. The central Z39.50/SRU gateway, which will be offered to those Z39.50 data providers that don’t have a local gateway, will partly solve this problem. This gateway relies on standard software modules, giving it a low level of complexity, which suggests that this component will be easy to maintain.

    5.3 Multiple requests are launched and processed in the browser

    Multiple requests transmitted to a set of servers are launched in the browser and the browser can handle the responses asynchronously. This means that major computation takes place within the browser’s environment, i.e.

    client side scripting is being applied to a major extent. This minimises the need for central processing power. Additionally the user can start browsing the results without waiting for the last server to have replied. With a central server the searches need to be aborted for slow servers when the user does not want to wait for them to respond.

    All these client side functions must be written in Javascript. Unfortunately different types of browser (particularly Internet Explorer and Netscape/Mozilla) implement some Javascript functions in different ways. In the worst case, different versions of Javascript code fragments have to be implemented for the different browser types that need to be supported. This is inconvenient and leads to complex scripts that might become difficult to maintain. Supporting many browser types will multiply the effort of debugging. The tools provided for


    IST 2000 25347/ Aimed solution for the TEL test portal debugging JavaScript are less developed compared to tools provided for server-side scripts. In general security

    settings have to be lax in order to perform client-side scripting.


Report this document

For any questions or suggestions please email