Mercury's Experiences with 3 Layer Architectures

 

Authors

Alasdair Arthur - email: 100016.1255@compuserve.com

Peter Burke - email: 70314.2335@compuserve.com

 

Version: 1.0b

November 1995


CONTENTS

1. Introduction

2. What is a Three-layer architecture (3LA) ?

3. So why would one want to use a 3LA ?

4. Why is MCL adopting a 3LA ?

5. 3LA Applications within MCL

6. MCL's Technical Architecture

7. IS organisational implications of a 3LA

8. Summary and Conclusions

Appendix A - Glossary of Terms


Disclaimer

This document was originally written at the end of 1995. It should be read as an historical snapshot of the experiences and views of its authors at that time. In particular, the capabilities and relative merits of the tools and techniques described are certain to change as time goes on, so any views expressed on such matters should be judged in the context of the period in which they were made: they may not still be valid when you read this!

The document was written by Alasdair Arthur and Peter Burke, both of whom since have left Mercury Communications Ltd. ("Mercury"). The views expressed here are theirs, and not necessarily those of Mercury.

The document remains in the public domain with Mercury's permission.


1. Introduction

During the last 18 months Mercury Communications Limited (MCL) have developed, tested and deployed a number of applications based on the three-layer architecture (3LA).

The purpose of this report is to detail MCL's experiences to date. It is not a highly technical document and is intended to be of use to Project Managers, analysts and developers in both IS Development and IS Operation groups.

This version of the document has been edited from the original internal one. The paper is divided into a number of sections

The paper is accompanied by a presentation. Both the presentation and this paper cover similar ground and can be read independently.

The views and opinions expressed in this paper reflect the views of the authors and do not necessarily reflect the views and opinions of MCL or C&W.

Peter Burke and Alasdair Arthur were technical architects at MCL. Both worked in the Strategy and Architecture Group.

Some of the words in this paper have been borrowed from other sources - particular thanks must go to Mike Gee (Captain America), Dick Keable (SRMS), PA Consulting (SSI & BQS) and Lloyd Burgess (PIN).

2. What is a Three-layer architecture (3LA) ?

2.1. A non-computing example.

The travel agent represents a real-world parallel to the concept of the three-layer architecture. In this example, the customer is equivalent to the front-end application, the agent to the middle-layer logic, and the various reservation systems to back-end IT systems.

Figure 1 - The Travel Agent

A customer makes a simple request to the agent, perhaps asking for the cheapest fare to a destination. The customer does not have direct access to airline guides or booking systems, but the agent does, and will transform the customer's query into a series of enquiries to different systems, compare and consolidate the results, and give the customer the answer. If a service changes (the advent of the Eurotunnel booking system for example) only the agent needs to be aware of this, the customer can be completely unaware of it and yet can still benefit from the services which it offers.

The key point is that the client knows what questions he wants to ask (how to get to Paris) but is completely unaware of the work that the agent is doing on his behalf.

2.2. Within a computing context.

It is important to state that the 3LA model describes an application framework for developing on-line transaction processing (OLTP) systems. Whilst the concepts of separating logic into presentation, business rules and data management components applies equally well to batch processing and MIS/Data Warehousing systems, the rest of this paper is directed towards the framework in which OLTP applications should be developed.

The Gartner Group separates the layers of a client-server application into presentation, function and data management. These functions can be partitioned in 5 different ways as illustrated in figure 2.

Figure 2 - The Gartner Definitions of Client Server Architectures

Depending on the technology chosen, an application may implement one or many of these scenarios.

A 3LA is usually taken as supporting models 2 & 3 (remote presentation/distributed function) and tending towards a 'thin client/thick server' model.

This should be distinguished from a thick client/thin server model typified by Oracle Version 6 which would be seen as a classic 'two layer model' and described as model 4 in the Gartner model.

Figure 3 compares a two and three layer models with the "traditional" mainframe model :

Figure 3 - Layered models compared

3. So why would one want to use a 3LA?

Rather than answer this question directly, it is better to start by asking what does a 3LA provide that a 2LA doesn't ?

Thus we to return to the original question - why would one want to use a 3LA? The simple answer is that one would choose to use a 3LA whenever more than one of the following conditions apply -

There is a more subtle advantage that a 3LA model has over a 2LA - namely that it supports the concept of the use of services rather than simply the accessing of data.

Typically in a 2LA environment the presentation layer must know exactly what data it needs, where it is and how to access it. The presentation layer will read and write to the database directly. Thus whenever the data changes, the presentation layer will need to be amended.

A 3LA model uses the concept of services (or agents in the previous analogy).

This concept of having an "agent" to act on behalf of applications is one of the powerful architectural ideas which is made possible by the three-layer architecture as opposed to a simple client/server design.

A service-based approach involves the creation of a set of processes which implement business functions which can then be used by any other process which needs that function. Examples of the functions involved include looking up a customer balance, updating address details, changing an employee's status, etc. This type of business logic has traditionally been built into individual applications, and has not been accessible from outside those applications.

In the 3LA, the presentation layer calls upon the business rules in the middle layer to perform specific requests (e.g. 'add a customer' or 'retrieve account balance'). The business rules determine how to perform the function 'retrieve account balance'. Consequently, when the processing to support the business rule changes then only the business rule code must be changed rather than the presentation layers that invoke the rule.

Figure 4 illustrates the concept of many applications sharing a range of services:

Figure 4 - Applications share common services

To summarise - The use of a service layer will allow applications to be insulated from the implementation of the business logic. It will also allow applications to be shielded from any changes to the underlying services, since only the services modules need to be altered.

The use of services explicitly requires that the applications are built within an API (Application Programming Interface) paradigm rather than a 'data-everywhere' paradigm. This idea needs to be explored in a little more detail.

Data-everywhere approach.

In this paradigm, the presentation layer is given access to data in the back-end systems. To make it easy for the presentation layer the data will usually be made available in an 'SQL-format' even if it is held as VSAM or DB2 (EDA/SQL is a good example of the 'data-everywhere' approach).

API approach.

In this paradigm the presentation layer has no direct access to back-end data. Instead, it calls upon a service (such as 'retrieve customer balance') without knowing or caring about where the data to support the request resides.

It is up to each organisation to determine which model they wish to follow. Within MCL we are very strong on the API approach - to such an extent that our newest 3LA application (BQS - discussed later) does not have any SQL calls from the desktop application, all access is provided through services invoked by an API.

4. Why is MCL adopting a 3LA?

It is true to say that writing an application using a 2LA model is currently easier than writing a 3LA model for the following three reasons:

The keys reasons why MCL is adopted a 3LA are discussed below -

a) Give the appearance of integration.

b) To support re-use across applications.

c) To provide consistent process logic.

d) To isolate users from the churn of functionality.

One way to solve problems (a) and (c) is to create an integrated application where data is only held once. In that scenario there is no requirement to give the appearance of integration (because there is only one system) and providing consistent process logic becomes less of an issue because the process logic will be simpler.

MCL have tried to go down this route and failed. The enormity of the task was beyond our capabilities. Instead we have fallen back onto a less ambitious goal of trying to isolate our users from our many systems whilst we gradually attempt to rationalise and consolidate our systems into a more consistent and coherent architecture.

Note that this is not to say that the integrated approach is wrong per se. For some companies with simple requirements and a straightforward systems environment, it may be the most cost-effective solution.

5. 3LA Applications within MCL

The 3LA applications within MCL follow a well-defined path. This section details the significant applications and discusses the lessons learnt.

Figure 5 - "Timeline" showing the evolution of 3LA in Mercury

5.1. Captain America

In November 1992 MCL initiated a project to assess some distributed computing software tools introduced to MCL by Hewlett-Packard. The HP proposition was that "these tools use Client-Server technology based around a UNIX computer and are particularly designed for rapid integration of existing (i.e. Legacy) systems into a new desktop application".

A prototype was developed with HP with the following objectives -

Technical Architecture

Figure 6 - Captain America Technical Architecture

Communication between the PC and UNIX middle layer was achieved using the Cambridge Technology Group (CTG) RPC toolset.

Access to the back-end systems was achieved in a number of ways -

Status

The application proved successful as a prototype and demonstrated that there were considerable benefits in hiding the back-end systems from the users.

The system never moved into production for the following reasons -

Conclusions

5.2. SRMS - Service Request Management System.

Captain America proved the concept and value of a 3LA and spawned two further projects - SRMS (developed during Autumn '93 to Spring '94) and 'The DCE Lab' (set up in Summer '93 and continued until Spring '95)

The lab was created to provide a small facility where we could demonstrate that the 3LA was workable and could be used as a framework to support a number of different hardware platforms.

Captain America had shown us that the screen-scraping tools were not the path we wished to follow for production use and the first task of the lab was to find a toolset that would be robust enough to support mission-critical systems

We used the lab to teach ourselves about a middleware technology called the Distributed Computing Environment (DCE) from the Open Software Foundation (OSF). We decided to use the OSF DCE product rather than alternative RPC products such as OEC and Netwise because of our wish to adopt a non-proprietary RPC mechanism and to prove operability across Sun, DEC, HP, RS6000 and PC platforms.

The purpose of SRMS was to provide a system that could support all the queries that a corporate customer might ring a call centre with. The idea was that a single phone number could be used to register all faults, problems and queries. This in turn meant that the CSA would need to be able to access data on many systems.

It was essential that the CSA not be required to learn the native interfaces to all the back-end systems and hence the ideas of Captain America proved useful.

Technical Architecture

Figure 7 - SRMS Technical Architecture

It was intended that the SRMS application use the DCE middleware to provide RPC communication between the various platforms. However, the middleware infrastructure was not yet established within Mercury so an alternative architecture was used.

This architecture consisted of a logical separation of the three layers rather than a physical one. The SRMS application communicated with back-end systems using screen-scraping on the PC via Dynamic Data Exchange (DDE) links.

When the users logged onto SRMS in the morning they also logged onto all the back-end systems. SRMS would then screen-scrape to obtain information from the back-end systems.

This approach had one large advantage, namely that each user individually logged onto the back-end system and thus the audit requirements of IS Security were fulfilled.

However there are a number of problems -

  1. Screen-scraping does not provide a robust solution. Care is needed to ensure that the SRMS application is made aware of any changes to the back-end screens
  2. The resources required to support several DDE sessions proved to be too much for Windows 3.11 to support, especially when ms-mail and Excel were running on the workstation
  3. The requirement to maintain a constant session with several back-end systems meant that any network glitches/hiccoughs directly affected the application. Often we would find that a session would hang causing us to reboot Windows.

As a consequence of 2) and 3), two actions were taken -

The upgrade to NT Workstation has proved extremely successful. Whereas previously under Windows 3.11 an SRMS session might have needed to be re-booted several times per day, under NT SRMS can go for several days without interruption.

The replacement of the DDE mechanism with DCE RPC links has proved more problematical - mainly due to the problems that removing direct logon to the back-end system caused with Security.

We had to link SRMS with the security system before we could remove the requirement for the users to individually log onto each of the back-end systems. This is turn meant that the DDE->DCE conversion had to wait until the security system had been developed.

Status

SRMS is being used for 300 users with the migration from Windows 3.11 (on a 486) to NT Workstation (on a Pentium) complete.

The DCE upgrade is being restarted now that IS Security have a security server that can be used within the 3LA framework (see later for an explanation of SSI - section 5.3).

Conclusions

5.3. SSI - Security Service Interface

Both Captain America and SRMS suffered from not having a well-defined security application that supported the concept of 'service authorisation' as opposed to data and/or platform authorisation.

The technical tools that we were using for the RPC mechanism (DCE) provides a security service (Kerberos) but we decided against using it, mainly because of the lack of tools available to manage it and because we wanted to reduce the amount of new technology which we were going to be introducing into the operational environment. Consequently, we took the decision to build a relatively simple 'home-grown' system, which is essentially similar in its approach to the Kerberos ticket-passing model.

This system is not intended to be our long-term solution but to provide sufficient functionality to enable our 3LA applications to go live and, just as importantly, to enable the developers and IS Operations to become completely familiar with the concept of 'service authorisation'.

The system had three objectives -

Technical Architecture

The technical architecture is a direct result of the existing infrastructure - the use of CA-ACF2 as the central security database is because this is our mainframe security package and the use of an LU6.2 communication protocol for the UNIX<->IBM connectivity is as a workaround for the absence of DCE on the mainframe. (We are currently on MVS V4.3, DCE is available on MVS V5.1).

Status

The SSI has been built and is ready to be used. It was intended that BQS be the first application to utilise the system but delays to BQS (in getting the back-end APIs written) have meant that we are looking at other options for moving the SSI from 'off-the-shelf' into production.

We anticipate enhancing SRMS to use the SSI over the next couple of months.

Conclusions

Implementing service based security using a simple 'home-grown' system rather than utilising the complexity of Kerberos has enabled everyone to concentrate on the paradigm shift that this requires rather than having to deal with the difficulties of getting Kerberos up and running.

Security now feel that they are 'in control' of the middle-layer services and are happy for 3LA applications to be developed and rolled out. The transition from the SSI to other security systems providing greater functionality is under the control of the Security department.

In the medium term we hope to phase in the DCE Kerberos security approach and phase out SSI.

5.4. BQS - Bill enQuiries Solution.

BQS is Mercury's first full 3LA application. Development started in April 1995 and it is currently (May 1996) in final testing. It is intended that the BQS application will be rolled out in mid 1996.

The following is an extract from the BQS statement of requirements -

"The Bill enQuiries Solution (BQS) will provide a seamless integration of existing and future front-office and billing systems, functionality and data. BQS will have a PC Windows 'look and feel' front-end, and a distributed architecture which will allow BQS to access functionality and data on other 'back-end' systems (e.g. SSBS, CARMS).

To the Customer Service Assistants (CSAs, or CAs) who will use BQS, none of the back-end systems will be visible, and therefore no knowledge of them will be required. If a CSA needs to see a copy of the customer's bill, they will not need to know where the bill is stored, or which system generated it.

To deliver the common interface and distributed architecture that BQS requires to shield users from the diversity of back-end systems, Mercury's recently adopted three-layer architecture will be used. This defines a middle layer that separates the users' desktop application layer from the back-end systems. The middle layer contains the knowledge and logic required for BQS to access the appropriate functions and data on the back-end systems and provide them in a consistent format for the desktop application to display to the users."

BQS is being developed by an external party (PA Consulting). In order to ensure that the application was developed to MCL standards and to reduce the amount of DCE knowledge PA had to acquire, we wrote a DCE toolkit that shielded the developers from having to know the low-level details of DCE.

We have ported this toolkit onto our VMS and HP-UX platforms and are in the process of porting the toolkit to the AIX platform. At the end of this exercise the toolkit will be available on all the non-PC DCE platforms that we use. This enables the DCE components of an application to be the same regardless of what platform they run on and reduces the learning time for any new developer.

Technical Architecture

Figure 10 - BQS Communications Architecture

The strategic objective of Mercury is to utilise TCP/IP and DCE on as many platforms as possible, thereby reducing the complexity of workstation connectivity. Other than direct connection to the IBM, communication between all BQS architectural components will be via DCE.

The combination of TCP/IP and DCE is available for all platforms other than the IBM machine running its present version of MVS. Hence an RS/6000 workstation will be used as a gateway to the IBM. The RS/6000 will run SNA LU6.2 to communicate with the IBM, and TCP/IP and DCE to communicate to the remainder of the system. The communication stacks for each of the three component platforms is shown in figure 10.

At a higher-level the technical architecture can be represented as in figure 11

Figure 11 - BQS Technical Architecture

Data Architecture

The Data Architecture for BQS reflects the use of a three layer architecture in that the vast majority of data used by the application is transitory, and BQS itself owns very little data. The implementation of the data architecture overlaid on the three layer architecture is summarised in figure 12.

Figure 12 - BQS Data Architecture

The vast majority of the data in BQS is merely in transit between the various back end systems and the presentation layer on the PC. The middle-layer database is used to store data before it is transmitted onwards, either to the presentation layer, or to the appropriate back end system(s). Data will be held in two different types of cache, store-and-forward and read-ahead.

BQS will attempt to write to the back end systems when requested to do so. However if a back-end system is unavailable, perhaps because it is down or in read-only mode, then BQS will store the write request in the Store and Forward Cache. This mechanism ensures that the Customer Service Assistant is not kept waiting for system writes to be performed.

The read-ahead cache is designed predominantly to improve the perception of system performance at the front end. It relies on the assumption that for a large proportion of enquiries, there is a degree of predictability in the way users access certain types of account information. For a high percentage of enquiries, CSA's typically need to view the account financial history, the bill and payment detail of the most recent n transactions, and the bill page information from the m most recent bills.

Once an account has been uniquely identified, and the initial details displayed to the CSA, then background calls can be made to the back end systems to populate the read-ahead cache with data relevant to the selected account. This can occur in parallel to the user navigating the data already presented at the front end. This illustrates an advantage of the three-layer model - work can go on in the background to gather data from many systems without affecting the user on their workstation.

When the CSA decides to view a bill page, for example, the read ahead cache is interrogated first for the required data. There are three possible outcomes of this interrogation:

The design of the middle-layer cache services has taken a considerable amount of time - far longer than was originally anticipated.

Conclusions

5.5. PIN - Provision the Intelligent Network

At the same time that IS were developing their 3LA application another area of MCL - Software Engineering and Operations Support Systems (SE/OSS) - was working in the same area. They had many of the same types of integration problems already described.

As a result SE/OSS initiated PIN (Provision the Intelligent Networks) in July 1994. The system was operational in December 1994 and provides a single provisioning interface between Front Office Systems and Intelligent Network Elements.

Technical Architecture

Figure 13 - PIN Technical Architecture

Prior to PIN, there was a manual process to support the interface between allocating a translated number (support by the TNAS system) and the update of the switch and notifying the billing system.

PIN removed the manual processes and provided a data-flow management facility.

The IN provisioning system is being moved between environments. This migration will be transparent to the users of TNAS. This illustrates one of the key advantages of the 3LA, in that it is able to insulate users from the change in back-end systems.

Due to the lack of DCE for the Macintosh, the Netwise product has been used for the TNAS->PIN interface. It is planned to migrate this to DCE once a suitable DCE product becomes available.

It should be noted that PIN has been implemented with screen-scraping technology. This was because the TANDEM-based package could not be modified. It does show perhaps the only area where screen-scraping can work - i.e. when the underlying application is changed infrequently and the changes are well documented and controlled.

5.6. Lessons Learnt.

a) Start with a prototype and use external people to help you.

It is only when you show users and management that you can bring together disparate systems together within a single presentation layer that they will believe it can work.

b) Don't go for the most complicated solution first.

For example, within DCE there are five components. We started by just using one part (RPC) and are now using three (RPC, CDS and DTS).

In a similar vein we have not implemented two-phase commit when managing updates to our back-end systems

Currently our business processes do not require that we support simultaneous updates but rather it is sufficient that if the data is duplicated then we have a process to ensure that the data is kept in step.

We will look at the use of a TP monitor (for example Encina) when the business need justifies it.

c) Prove to yourself that screen-scraping is not a long-term solution for production systems.

You will be told that screen-scraping is easy to use and is stable.

We have found that the change control to ensure that the applications that 'scrape' are notified when a screen is changed proved unworkable

d) Don't underestimate the change in philosophy that a 3LA architecture requires.

This is discussed in more detail in a later section.

e) Work through the implications of the back-end systems being unavailable.

For instance, if the purpose of the application is to shield users from knowing about the billing system, what happens when you want to update data on the billing system but the system is in read-only mode?

We have implemented a store-and-forward cache but this may be over-complicated for your requirements.

6. MCL's Technical Architecture

This section will describe the technical components that MCL is using for BQS. BQS is the model of our medium-term technical architecture.

6.1. Desktop

GUI Tool Powerbuilder

OS Windows 3.11 or NT Workstation

DCE PC-DCE from Gradient

Protocol TCP/IP

Note:

1. The communication between Powerbuilder and PC-DCE is via a set of defined DLL's. This was a deliberate decision so that any application that can invoke a DLL can reuse the middle-layer functionality even if it doesn't support RPC's (e.g. Excel)

2. There is no SQL on the desktop. All communication will be via RPC.

3. We are currently in the final stages of testing whether BQS will run satisfactorily under Windows 3.11. The load on the OS created by BQS will be less than by SRMS (SRMS requires DDE and SQL*Net) which means that the application should be able to run under Windows 3.11, although the application will be rolled on Windows NT Workstation when budgets allow.

4. The use of Powerbuilder for BQS was based on the fact that it was used for SRMS.

One of the key strengths of Powerbuilder is the ease by which you can populate an application window from an Oracle table. This feature is not used within BQS. Consequently, if we were going to revisit our choice of GUI front end for our 3LA applications this feature would be less important.

6.2. Middle-layer

OS HP-UX Version 9.0

Language C, Pro*C

Database Oracle V7

DCE From HP

Protocol TCP/IP

NB

  1. Oracle is used within the middle-layer to hold the read-ahead cache and the store-and-forward cache. BQS does not hold any permanent data within its Oracle database.
  2. The middle-layer services will, where ever possible, not access data directly. They will call upon back-end routines to do this for them. This supports the principle of hiding the data wherever possible.
  3. In those cases where the middle-layer needs to access a remote Oracle database then it will do this by transporting the request as an RPC and issuing the SQL locally. This is because DCE/RPC is more efficient as a transport mechanism then SQL*Net.

6.3. VAX Access

OS VAX OpenVMS V5.5

DCE Digital DCE

Languages C, FORTRAN, Pro*Fortran and Pro*C

Datastores Oracle, RMS

Transport TCP/IP

NB

  1. Access to RMS data is supported using FORTRAN routines. Thus an RPC request received by the VAX will be interpreted by C code which will call a FORTRAN routine to read the RMS data.
  2. Because of MCL's historical use of FORTRAN the initial 3GL Oracle language was Pro*Fortran. Where 3GL code is required in new applications then this will be written in C.
  3. As with the middle-layer, there is no need for SQL*Net on the VAX to support BQS since all SQL calls will be made local to the database.

6.4. IBM Access

Ideally, we would access the IBM in the same manner as the other platforms using DCE as our transport protocol.

DCE is supported under MVS 5.1 (Open Edition). However, we run MVS 4.3 and the effort involved in upgrading the mainframe is prohibitive. Consequently we have implemented a workaround by using APPC (Application Peer to Peer Communication) between a UNIX box and the IBM.

MCL has no experience writing APPC programs so we looked around for help. Fortunately, Oracle provide a gateway tool (PGA - Procedural Gateway for APPC) that enables one to invoke an Oracle stored procedure and Oracle to issue the APPC call that invokes the CICS code on the mainframe.

Figure 14 illustrates how access to the mainframe is achieved.

Figure 14 - Technical Architecture for accessing mainframe

NB

  1. The middle-layer doesn't know that DCE isn't running on the mainframe - it just invokes an RPC call. The fact that the RPC is serviced by code running on an RS6000 is transparent to the middle-layer service.Thus when we upgrade to MVS 5.1 the middle-layer RPC's will not need to change. We will simply implement the RS6000 code that receives the middle-layer RPC on the mainframe.
  2. In order for us to eventually migrate from the PGA workaround to the MVS 5.1 solution we have taken the deliberate decision not to store any data (e.g. for the read-ahead and store-and-forward caches) on the Oracle database on the RS6000. The RS6000 is used solely as a gateway to the mainframe.
  3. At first glance the architecture seems to have a large number of components which would suggest that performance would be poor. However, initial tests show that the performance is acceptable for applications where real-time access to back-end functions and data is required. The read-ahead cache that we have built for BQS will further isolate the CSA from the elapsed time involved in accessing the mainframe.
  4. A simpler architecture for providing communication to the mainframe would be likely to provide more resilient access (on the assumption that the less pieces of software and hardware are involved the more likely it is that it will work). To try and alleviate this concern we will be using two RS6000/Oracle PGA for mainframe access.

7. IS organisational implications of a 3LA

Normally, the adoption of any new technology will have some affect upon the IS organisation. Sometimes the affect is small (e.g. using COBOL/2 rather than COBOL/1) and occasionally it is larger (e.g. the adoption of a CASE tool for the development lifecycle).

Whilst it is not imperative that the IS organisation changes to support a 3LA, to gain the full benefits it is essential. This section outlines some of the reasons why change is required and what MCL's approach is.

a) IS groups will use each other's functions

Life was simple when an IT group was closely aligned with a business group (e.g. billing or sales). The users told the IS group what to do and the developers implemented systems using the data owned by the users/IS group.

However, these days life is more complicated and systems are required to cover many functional areas. For example, account managers will want access to a customer balance even though this information 'belongs' to the credit and collection department. And furthermore the account manager does not want to have to know about the new credit and collection system that has just been implemented!

The implication of this is that each IS group has a responsibility to provide access to the functions and services that it is the steward for. In practice this means that the local IS group must "publish" the services which other groups must call. The agreement must be that the local IS group will not unilaterally change the service definition (i.e. the API) even if the internal data structure changes (e.g. account balance moves from a VSAM to a DB2 datastore).

There are implications here for the analysis and design phases of the lifecycle too. It will often be necessary for two or more projects to agree upon the definition of a service which they are to share. This means that the organisation must provide a forum for such discussions, some means of brokering them, and a process for compliance approval to ensure that reuse actually takes place.

b) API's need to be published.

It has been said that, on average, an IS person will spend 10 minutes seeing if there is something they can reuse before they start developing it for themselves.

Since one of the ideas behind the 3LA is that the middle-layer functions can be re-usable, it is imperative that the services that are written are easily accessible and widely usable.

This implies the need for a librarian who not only ensures that people know what services are available (i.e. effective and efficient communication of both the business functionality supported by the service but the low-level API definitions as well) but also ensures that when a service definition is proposed then it is defined so that it can be reused - i.e. it is not built solely for the convenience of the first group who uses it.

This has a further knock-on effect... how do you define what the difference is between a 'good' and a 'bad' service definition?

The program for re-use must be pro-active with development staff and project managers being rewarded for re-using software as opposed to getting paid by the number of lines of code or number of modules that are created.

It is probably fair to say that MCL has not made much progress in the establishment of a 'pro-active re-use program'. Whilst the number of 3LA projects is small and 'everyone knows everyone else' then this is not too much of a problem. However, it is an area that we are actively working on improving.

c) Testing is more difficult and challenging.

The 3LA enables desktop applications to gather data from many data sources on many differing platforms into one application. At some stage the application must be tested and this means that it is no longer sufficient to have 'billing' and 'ordering' testing teams or a 'mainframe' and 'VAX' testing split.

The 3LA implies that testing needs to be cross-functional and cross-platform if it is to be effective and not be a bottleneck in the development lifecycle.

d) Developers can specialise.

The 3LA is not all bad news for the IS organisation! By splitting an application into explicit presentation, business logic and data access functions is it possible for different groups to work on different areas. For example, just because an application is moved from CICS to a GUI doesn't mean that your CICS programmers are no longer required or need to be retrained on the GUI. The CICS programmers will be needed to create back-end services that the presentation layer will invoke.

Similarly, the GUI developers need not be concerned with writing efficient SQL or understanding the RMS data structure since they will be talking to be middle layer via RPCs that reflect a level of business functionality.

e) Operating a 3LA is more complicated than a mainframe architecture.

A simple two-layer client-server architecture (e.g. PC's running against a RDBMS on a server) is more complicated to run than a MVS/CICS application. Facilities that one takes for granted on a mainframe such as system and data management/monitoring tools and scheduling tools are less mature within the client/server arena than within the mainframe environment.

Having an extra level of hardware (the middle-layer) only exaggerates the difference. If you aren't running an efficient and effective 2-layer client/server operation it is unlikely that you will be able to operate your 3LA applications successfully.

The support of the application is also more complicated than for an application based on a 'simple' single-tier architecture. The application will be running on many platforms and interacting with many applications and the old model of having a support group per platform (e.g. VAX support and IBM support) or per application (e.g. Billing support and Sales support) will not be flexible enough to support the 3LA applications.

MCL have adopted an holistic approach for the support of SRMS with one team taking ownership of all the problems, regardless of whether they are application, platform or network.

f) Security control is by service not by data element or application.

Typically, IS Security operate by allowing people to access platforms, applications or data elements. However the 3LA removes the direct connection between users and back end applications. For example, when an application integrates data from a back-end billing system and a credit and collection system it no longer makes any sense to ask the user to directly log onto systems whose screens they will never see or use.

Thus the 3LA requires that security is based at a service level. A user gains authorisation to use a service (e.g. 'update customer address') and the service then acts as an agent of the user and performs whatever functions 'update customer address' requires.

To summarise - a user is granted authorisation to use a service and the service is granted access to update/read files as appropriate.

In the long run this actually makes the security of the back-end systems easier. This is because there will usually be fewer services than there are users and the turnover rate of services will be less than that of employees.

8. Summary and Conclusions

a) Implementing a 3LA is not simple. The technology is not yet at the stage where it can be employed 'straight out of the box'. You will need to employ good low-level technical people if you want to succeed.

b) As with any new technology, you need to be sure that the benefits outweigh the risks. If you can afford to wait six months do so (and in six months time ask the same question again).

c) The technology is a means not an end. It is easy to get drawn into the technology without appreciating the wider aspects.

In fact, within MCL we feel that we have the technology under control and that the success or failure of the 3LA architecture is more dependant on organisational issues than anything else.

d) Make sure that you start small. Start with a pilot to ensure that the technology fits your skills and that your organisation is capable of dealing with the changes that the 3LA will bring.

e) Don't convert an entire 'legacy' application to a 3LA overnight.

Start by creating services 'on-demand' when they are required. It is perfectly reasonable for some users to continue to access the back-end systems via 3270 terminals whilst other users are given a GUI presentation.

f) Use your legacy functions wherever you can. Try and create wrappers around the existing code rather than rewriting all the code onto a new platform.

g) Performance of a 3LA is surprisingly good. Whilst first impressions might suggest that 3LA means lots of connections between lots of boxes, all that is happening is that a 3LA makes explicit what is happening in the background on a 2LA.

Expect your 3LA application to perform as well as your 2LA and to scale up better.

h) A 3LA gives you a platform to implement time-dependant processing (e.g. only update the billing system when the provisioning system has accepted the order). In a 2LA model this type of processing is difficult to implement.

i) Don't underestimate the culture change that the 3LA will require. It will take people longer than you realise to understand the implications of a 3LA on the way they design and implement systems.

j) Be aware that 're-use' will not happen by accident, you will need to put in a program to ensure it happens.

k) And finally..... remember that "in a journey of a thousand miles the first step is the hardest".

Appendix A - Glossary of Terms

Advanced program-to-program communications (APPC)

High level communications protocol from IBM that allows a program to interact with another program. It provides commands for managing a session, sending and receiving data and transaction security and integrity (two-phase commit).

APPC has only supported SNA, utilising SNA's LU 6.2 protocol for session establishment and thus APPC and LU 6.2 have been considered synonymous. In time, APPC will support other industry protocols, such as TCP/IP and OSI.

Application programming interface (API)

The interface (calling conventions) by which an application program accesses other services. An API is defined at source code level and provides a level of abstraction between applications to ensure the portability of the code. The API's main task may be the translation of parameter lists from one format to another and the interpretation of call arguments in one or both directions.

Asynchronous call

A service request that does not wait for a reply but instead continues processing until a command is encountered to get the reply.

Atomicity

Property of a transaction in which it will either complete as a whole unit of work or be rolled back.

CICS

Customer Information Control System - an IBM Transaction Processing Monitor which is used on IBM mainframe systems to efficiently manage resources when many users are using online transaction.

Client

End-user processor; usually a workstation or high-performance PC that supports a graphical user interface (GUI).

Commit

Permanent completion of a transaction component. At commit, the information is guaranteed to be consistent and can be made available to other system users.

Concurrency Control

Characteristic of a TP monitor that ensures that concurrent transactions will not have the same record out for update simultaneously. This feature allows multiple users to execute database transactions simultaneously without interfering with each other.

Consistency

A property of transactions that implies that the results preserve data integrity; in other words, given the same conditions, a transaction must always produce the same results.

Daemon

A background utility program that is automatically invoked when needed; usually associated with UNIX systems.

Distributed Computing Environment (DCE)

An architecture consisting of standard programming interfaces, conventions and server functionalities (eg. naming (CDS), security (Kerberos), distributed file system (DFS), remote procedure call (RPC)) for distributing applications transparently across networks of heterogeneous computers. DCE is promoted and controlled by the Open Software Foundation (OSF).

Distributed data

Data that is physically separated between platforms.

Distributed transaction processing (DTP)

Transaction processing running on more than one server across a network.

Durability

Property of a transaction that ensures committed transactions will tolerate system failures.

Encina Monitor

Development and execution environment for programming and integrating distributed TP applications. Provides security, load balancing and scheduling on heterogeneous platforms. Provides systems administrators with tools to configure and manage Encina applications.

Graphical User Interface (GUI)

Typically a GUI allows several programs to be displayed in separate windows on the screen. The operator can move these windows round, change their sizes, and easily switch from one program to another. The programs have a consistent user interface that makes use of pull-down menus, dialogue boxes, and graphical objects such as icons, scroll bars, and buttons.

Isolation

Property of a transaction in which its components are executed independently of one another.

Kerberos

Security system developed at MIT that authenticates users. It does not provide authorisation of services or databases; it establishes identity at logon, which is used throughout the session.

LU 6.2

SNA protocol that establishes a session between two programs. Before LU 6.2 processing was done only in the mainframe. LU 6.2 allows processing to take place at both ends of the communications, necessary for today's distributed computing and client/server environment. (See APPC).

Middleware

Software that mediates between an application program and a network. It manages the interaction between disparate applications across the heterogeneous computing platforms. DCE software that manages communication between distributed applications is an example of middleware.

Multithreading

Allows a single process to execute multiple subroutines (threads) concurrently without starting a separate process. Because these threads access the same address space as their parent process, they can communicate with each other more effectively than if they were separate processes.

Node

Term used to describe a single computer system or network component; also describes systems that appear to be a single computer at the operating system interface level: for example, multiple processors included in a tightly coupled system.

On-Line Transaction Processing (OLTP)

The processing of transactions by computers in real time.

Open Software Foundation (OSF)

A consortium of vendors, service providers and end-user companies formed to promote open computing. The OSF defines the standards for technologies such as DCE and produces source code which is then licensed by vendors. Some of the leading sponsor members of OSF include Digital Equipment Corp., Hewlett Packard, IBM, Hitachi, Sun Microsystems, Bull and Transarc.

Oracle

Relational DBMS from Oracle Corp., Redwood Shores, CA, that runs on a wide variety of computers from micro to mainframe. It was the first DBMS to incorporate the SQL language.

PowerBuilder

Application generator for developing Windows client/server applications from Powersoft Corporation, Burlington, MA. It uses a programming language called PowerScript that is similar of BASIC. PowerBuilder supports SQL and several databases, including DB2 and Oracle.

PowerMaker is a subset of PowerBuilder with a simplified interface for power users and departmental use. It accesses popular databases as well as its own relational database. PowerViewer is the query, reporting and business graphics generator for both products.

Relational Database Management System (RDBMS)

A data storage mechanism characterised by the use of normalised tables of data which can be freely related (or "joined") to give many different perspectives on the data, and thus great flexibility.

Remote procedure call (RPC)

A mechanism used to invoke functions that are not locally available to a process; typically used for implementing server calls from a client.

Rollback

Reversing the results of a partially completed transaction.

Record Management Services (RMS)

A Digital disk file storage method which provides programs with random access to data.

Screen Scraper

A piece of software used to automate interaction between two computer systems through the terminal interface (designed for human use) of one of those systems. Typically, the screen scraper interacts with terminal emulation software to generate input to and process output from the "host" system through terminal screens.

Systems Application Architecture (SAA)

IBM plan to create software to weave together its incompatible computers. According to IBM advertising, "SAA will make it possible for everyone in an organisation to access information regardless of its location. What's more, all software written to SAA specifications will provide similar screen layouts, menus and terminology".

Server

Back-end processor for a department or whole enterprise that supports multiple users; provides specific information or processing resources to a user community. Servers range from a PC to a mainframe.

Structured Query Language (SQL)

Invented by IBM, SQL is a powerful database language used for creating, maintaining and viewing database data.

Systems Network Architecture (SNA)

IBM mainframe network standards. Originally a centralised architecture with a host computer controlling many terminals, enhancements, such as APPC (LU 6.2), have adapted SNA to today's peer-to-peer communications and distributed computing environment

Synchronous call

A type of service call in which the client sends the service request and waits (blocks) until the service response is received before performing any other processing.

Syncpoint

The point in the transaction during which it is either committed or rolled back.

TCP/IP

Transmission Control Protocol/Internet Protocol; standard UNIX and Internet communications protocol.

Terminal emulator

A program that allows a computer to act like a terminal, eg. a vt-100. The computer thus appears as a terminal to the host computer and accepts the same escape sequences for functions such as cursor positioning and clearing the screen.

Thread

An individual subroutine spawned by a parent process that can be executed simultaneously with other spawned subroutines. Each of these subroutines shares the address space of the parent process.

Transaction

A logical unit of work that exhibits the ACID properties: atomicity, consistency, isolation, and durability.

Transaction processing monitor (TP monitor)

A communications control program that efficiently manages transactions that transfer data between multiple, possibly distributed applications. TP monitors provide the properties of atomicity, consistency, isolation, and durability.

Transparency

Feature of some client/server architectures that allows a client to call a server without knowing its physical location. Transparency allows servers to be physically moved throughout the system without having to modify the client. Also referred to as implicit service routing.

Two-phase commit

A process by which consistency is maintained across data sources. In the first phase, the prepare phase, all participants (databases or other resources managers) in a transaction are polled to see if they are prepared to commit a transaction. In the second phase, the commit phase, if all of the participants responded affirmatively to the prepare phase, the transaction will be committed; on the other hand, if any of the participants responded negatively, the transaction will be rolled back.

UNIX

A powerful and complex operating system for computers for running data processing. UNIX provides multitasking, multi-user capabilities that allow both multiple programs to be run simultaneously and multiple users to use a single computer.

Virtual Memory System (VMS)

Multi-user, multitasking, virtual memory operating system for the VAX series from Digital. VMS applications will run on any VAX from the MicroVAX to the largest VAX.

Virtual Sequential Access Method (VSAM)

An IBM disk file storage method which provides programs with random access to data.

Virtual Telecommunications Access Method (VTAM)

Software that controls communications in an IBM SNA environment. It supports a wide variety of network protocols, including SDLC and Token Ring. VTAM can be thought of as the network operating system of SNA.

--- End of Document ---


©Mercury Communications Ltd 1996


Alasdair Arthur's pictureAlasdair Arthur Peter Burke's picturePeter Burke


The page's web-counter says that you are visitor number

since 9th May1996.