Datatrieve-20 project
TOPS-20 Dynamic Library Requirements
Author: David Dyer-Bennet
MRO1-2/L14
DTN 231-4076
DYER-BENNET AT KL2102, MRVAX::DDB
Edition: 1.4, 7-Dec-83
File: DYN1R.MEM
This document exists online as
KL2102::EXODUS:<DYER-BENNET.PUBLIC>DYN1R.MEM
TOPS-20 Dynamic Library Requirements Page ii
Preface
Preface
This is a component software product requirements document. It should
clearly and specifically define the technical requirements of the
product which is being developed. This document forms the baseline from
which the Specification as well as the cost and schedule estimates are
prepared in Phase 1.
Issue History
Issue 1.0 29-Jul-83 First public release
Issue 1.1 2-Aug-83 Incorporate informal review comments
Issue 1.2 11-Aug-83 Incorporate inspection comments; add trap and
PSI requirements
Issue 1.3 22-Sep-83 Incorporate informal review comments
Issue 1.4 7-Dec-83 Incorporate inspection comments; Currently in
development
TOPS-20 Dynamic Library Requirements Page iii
Table of Contents
Table of Contents
1.0 PRODUCT SUMMARY . . . . . . . . . . . . . . . . . . 1
2.0 TERMINOLOGY AND CONVENTIONS . . . . . . . . . . . . 2
3.0 ENVIRONMENT . . . . . . . . . . . . . . . . . . . . 2
3.1 Users . . . . . . . . . . . . . . . . . . . . . . 3
3.2 Hardware . . . . . . . . . . . . . . . . . . . . . 3
3.3 Software . . . . . . . . . . . . . . . . . . . . . 3
3.3.1 Other Software Required For DYNLIB . . . . . . . . 3
3.3.2 Other Software That Requires DYNLIB . . . . . . . 3
3.4 Services . . . . . . . . . . . . . . . . . . . . . 4
4.0 SOFTWARE CAPABILITIES . . . . . . . . . . . . . . . 4
4.1 DYNLIB Capabilities . . . . . . . . . . . . . . . 4
4.2 Writing A Dynamic Library . . . . . . . . . . . . 5
4.2.1 Passing Control And Returning . . . . . . . . . . 5
4.2.2 Memory Allocation . . . . . . . . . . . . . . . . 6
4.2.3 PSI And Trap Management . . . . . . . . . . . . . 6
4.2.4 Error Handling . . . . . . . . . . . . . . . . . . 8
4.2.5 Initialization . . . . . . . . . . . . . . . . . . 8
4.2.6 Debugging . . . . . . . . . . . . . . . . . . . . 9
4.3 De-linking . . . . . . . . . . . . . . . . . . . . 9
4.4 "Streaming" . . . . . . . . . . . . . . . . . . . 9
4.5 Library Version Checking . . . . . . . . . . . . 10
4.6 DYNLIB Version Checking . . . . . . . . . . . . 10
5.0 PUBLICATIONS . . . . . . . . . . . . . . . . . . . 11
6.0 PACKAGING . . . . . . . . . . . . . . . . . . . . 11
7.0 INSTALLABILITY . . . . . . . . . . . . . . . . . . 11
8.0 EASE OF USE . . . . . . . . . . . . . . . . . . . 12
9.0 PERFORMANCE . . . . . . . . . . . . . . . . . . . 12
10.0 RELIABILITY . . . . . . . . . . . . . . . . . . . 13
11.0 MAINTAINABILITY . . . . . . . . . . . . . . . . . 14
12.0 MAINTENANCE . . . . . . . . . . . . . . . . . . . 14
13.0 COMPATIBILITY . . . . . . . . . . . . . . . . . . 14
13.1 Compatibility With Existing Libraries . . . . . 15
13.2 Product Compatibility . . . . . . . . . . . . . 15
13.2.1 Dependency Issues . . . . . . . . . . . . . . . 15
13.3 Standards Conformance . . . . . . . . . . . . . 16
13.4 Internationalization . . . . . . . . . . . . . . 16
14.0 EVOLVABILITY . . . . . . . . . . . . . . . . . . . 16
15.0 COSTS . . . . . . . . . . . . . . . . . . . . . . 17
16.0 TIMELINESS . . . . . . . . . . . . . . . . . . . . 17
17.0 CONSTRAINTS AND TRADES-OFF . . . . . . . . . . . . 17
18.0 APPROVAL PROCESS . . . . . . . . . . . . . . . . . 17
APPENDIX A PROBLEM PRIORITIES
TOPS-20 Dynamic Library Requirements Page 1
PRODUCT SUMMARY
1.0 PRODUCT SUMMARY
A "routine library" (often called simply a "library") is a group of
routines and data which is intended to provide services to its caller.
A routine library is also sometimes called a "package."
On TOPS-20, the most familiar form of library is probably the "REL
library" as understood by LINK and MAKLIB. In this form of library, the
association between addresses exported from the library (such as routine
entry points) and references to those addresses in the caller (for
example, routine calls) is made by the linker, at link time.
A "dynamically linked library" (often called simply a "dynamic library"
in this document) is a library which is merged into a program on request
at execution time. The assocation between exported addresses and
references is made dynamically at that time.
This document states the requirements for a dynamic library facility for
TOPS-20. This facility will include code to support the dynamic linking
functions described above, support facilities such as .UNV files and
.L36 files to simplify use of the functions, documentation, and rules
for use.
The dynamic library facility described in this document should, if
implemented, provide the following benefits compared to continued use of
REL libraries in the current style:
o Provide a discipline for independently-developed packages to
share a process peacefully while providing access to the
resources they need to do their jobs (resources that require
discipline to share successfully include address space, the APR
trap system, and the software interrupt system).
o Easier software updates. A new version of a package
implemented as a dynamic library can be introduced into all
programs calling it by simply placing it on the directory from
which dynamic libraries are loaded. There is no need to relink
programs using it.
o More consistent program behavior. If a facility is always
provided by the same package of code, then it will always be
provided in the same way.
o More efficient use of physical memory and swapping space. If a
facility provided by a dynamic library is never called during a
run of a program, that library will never become part of that
process' address space. If several programs are using the same
dynamic library, they will share a single copy of its "pure"
parts.
o Lower cost of engineering other products. Using an existing
package to provide a service is much cheaper than having to
modify it or write one from scratch.
TOPS-20 Dynamic Library Requirements Page 2
TERMINOLOGY AND CONVENTIONS
2.0 TERMINOLOGY AND CONVENTIONS
This document exists primarily to specify requirements for the "dynamic
library mechanism," which we define as "that which causes a library to
be merged into a program on request, and performs other functions
necessary to make that useful." To avoid confusion, I will from now on
call this "DYNLIB." In the rest of this document, when I refer to a
"dynamic library" I am referring generically or specifically to some
library designed to be called through the services provided by DYNLIB.
A requirement of DYNLIB will always appear in a paragraph by itself, and
will begin with "REQUIREMENT:".
Goals of DYNLIB of lower priority than a requirement will be listed in
the same format as requirements, substituting "GOAL:" for
"REQUIREMENT:".
Sometimes, due to our experience with the DYNLIB prototype or for other
reasons, we may think that a goal or requirement is not fully
attainable. A paragraph beginning "RESTRICTION:" will describe parts of
requirements or goals that may not be attainable. The purpose of this
is to give a realistic set of requirements which can be met.
NOTE
A restriction is not a requirement. That
is, the presence of a restriction in this
document is not a demand that that
restriction be placed in the final
product. Rather, it is a statement that
that restriction is acceptable in the
final product if it is found to be
necessary during phases 1 and 2.
Related to the DYNLIB requirements are some rules that must be followed
by writers of libraries to be called through DYNLIB. These appear in
this document in a paragraph by themselves, beginning with "DYNLIB use
rule:". Many more rules than appear here will be necessary; only the
ones following directly from these requirements are included here. The
full set of rules for writing a dynamic library will be documented as
part of the development effort of this project.
3.0 ENVIRONMENT
TOPS-20 Dynamic Library Requirements Page 3
ENVIRONMENT
3.1 Users
Dynamic libraries will be written primarily by Digital engineers.
Moderately sophisticated customer system programmers may occassionally
be called upon to write dynamic libraries, as may Digital software
specialists.
GOAL: Dynamic libraries will (eventually) be used implicitly by most
higher-level language programs, but this must be transparent to
customers using those languages.
Both application and system programs are likely to invoke dynamic
libraries explicitly.
Digital-written utilities will also use dynamic libraries extensively.
3.2 Hardware
REQUIREMENT: DYNLIB must run on a PDP-10 family processor with extended
addressing (i.e. KL model B). KS processors need not be supported.
REQUIREMENT: DYNLIB must run on microcode version 326 and later.
As described below, DYNLIB will encourage the use of extended
addressing. Some dynamic libraries may discover microcode and monitor
bugs relating to extended addressing. Anticipating and fixing these
bugs is not part of the development of DYNLIB.
3.3 Software
3.3.1 Other Software Required For DYNLIB
REQUIREMENT: DYNLIB must run on TOPS-20 release 5.1 and later.
In a future release of the operating system, support for DYNLIB should
be built in. This is discussed under Evolvability. Operating system
support of DYNLIB is not a goal of this project.
3.3.2 Other Software That Requires DYNLIB
DYNLIB is required for Datatrieve-20 V1.0, dynamic extended RMS (V3),
and dynamic callable DBCS (V7).
All other software projects should consider if they would benefit from
using DYNLIB. In particular...
TOPS-20 Dynamic Library Requirements Page 4
ENVIRONMENT
EXTERNAL REQUIREMENT: Projects implementing RMS access for the first
time should call RMS as a dynamic library.
3.4 Services
No special services are required for DYNLIB.
4.0 SOFTWARE CAPABILITIES
4.1 DYNLIB Capabilities
In any library, the addresses of some objects in the library must be
made available for use outside the library. Routine entry point
addresses and, more rarely, addresses of data locations must be
"exported" from the library. In REL libraries, this is done using
global symbols. In VAX/VMS sharable images, this is done using what
they call "universal" symbols (more global than global symbols).
REQUIREMENT: It must be possible to export the addresses of routine
entry points and data locations from a dynamic library to its caller.
REQUIREMENT: It must be possible to call a dynamic library from a
non-zero section.
REQUIREMENT: It must be possible to call a dynamic library from section
0. This is provided ONLY to ease conversion of existing programs to use
of dynamic libraries. See Performance, below
RESTRICTION: From section 0, it need not be possible to refer to
addresses exported from a dynamic library in any manner other than a
routine call.
REQUIREMENT: It dynamic libraries must work in non-zero sections.
RESTRICTION: It is not necessary for dynamic libraries to work in
section 0.
REQUIREMENT: The contents of a dynamic library must not be part of the
address space of its caller until a reference is made to an address
exported from that library. (This requirement simply insists that a
dynamic library be in fact dynamic.)
REQUIREMENT: Using an address exported from a dynamic library must be
as similar as possible to using an address exported from a REL library.
RESTRICTION: Since DYNLIB must support extended addressing, and since
the location in memory of a dynamic library may not be known at link
time, it is acceptable to require that references to addresses exported
from dynamic libraries be indirect through a pointer supplied by the
dynamic library or DYNLIB.
TOPS-20 Dynamic Library Requirements Page 5
SOFTWARE CAPABILITIES
RESTRICTION: It is not necessary for DYNLIB to work in execute-only
programs or with execute-only libraries. Because DYNLIB will work by
merging EXE files into the running program, it will not work with the
current implementation of execute-only.
4.2 Writing A Dynamic Library
It would be nice to be able to write dynamic libraries in any language.
There are several reasons why this cannot reasonably be made a
requirement:
o Many languages do not produce code that will run in a non-zero
section.
o Most language OTS's will not run in a non-zero section.
o Most language OTS's think they own the entire process they
reside in. If dynamic libraries were written in these
languages, the OTS in one dynamic library would quite probably
conflict with the OTS in another dynamic library, even if they
were written in the same language. Memory management, the
software interrupt system, and APR trapping are likely areas of
conflict.
REQUIREMENT: it must be easy to write dynamic libraries in BLISS and
MACRO.
4.2.1 Passing Control And Returning
REQUIREMENT: Invoking a dynamic library must be made as transparent as
possible (invoking the dynamic library should look as much as possible
like invoking a routine from a REL library). No special subroutine call
may be required to load a dynamic library. The library must be loaded
as a side-effect of referring to an address exported from it.
RESTRICTION: Before monitor support for DYNLIB is implemented, it is
acceptable to require that a routine in any given dynamic library be
called before referring to an address exported from that library by any
method other than a routine call. This restriction is necessary if
products using DYNLIB are to ship before the next major monitor release.
RESTRICTION: It is acceptable to require that calls to routines in
dynamic libraries be made with the PUSHJ instruction and that the
routines return with the POPJ instruction.
It is not acceptable for a dynamic library to insist that it be mapped
in at a fixed address. If it were done, there is every possibility that
two dynamic libraries requiring the same address would be called from
the same program. (Consider the problems already encountered in sharing
TOPS-20 Dynamic Library Requirements Page 6
SOFTWARE CAPABILITIES
section zero between a user program, DDT, PA1050, FOROTS, COBOTS, and
RMS.) Since one of the goals of DYNLIB is to encourage using dynamic
libraries as building blocks when constructing a program, this is
unacceptable.
On a PDP-10, the hardware makes no good provision for writing position
independent code. However, it is easy to write section independent code
that will run happily in any non-zero section.
RESTRICTION: It is acceptable to assign an entire section to each
dynamic library invoked. On a KL, this limits a program to using at
most 30 dynamic libraries.
4.2.2 Memory Allocation
Memory is a resource that must be obtainable dynamically by all of the
parts of a program, including all dynamic libraries which that program
calls.
REQUIREMENT: DYNLIB must own the entire address space of any process
which calls dynamic libraries. DYNLIB must provide facilities for
dynamic libraries and main programs to allocate address space in units
of whole sections, pages, and words.
RESTRICTION: If, as suggested above, DYNLIB assigns each dynamic
library its own section, DYNLIB need supply only a section allocator and
a rule saying that the memory in each section must be managed by the
program or library which created it or was placed into it by DYNLIB.
GOAL: There should be some user control of section allocation.
EXTERNAL REQUIREMENT: The TOPS-20 monitor must not create sections
automatically when pages in nonexistent sections are written to.
Creating these sections automatically would mask errors in section
allocation or data reference.
4.2.3 PSI And Trap Management
The APR trap system and the software interrupt (PSI) system are unique
resources that must somehow be shared among all the packages of code
making up a program. For example, Datatrieve wants to trap all
arithmetic exceptions occurring while it executes, and issue appropriate
messages to the user. When Datatrieve calls MTHLIB, however, MTHLIB
wants to trap the overflows itself and perform fixups. This sort of
layered use of these unique resources must be possible.
Some channels of the PSI system have fixed meanings. These are
essentially the same as traps. Some events, such as characters typed at
the terminal, can be assigned to any available PSI channel. These
events are also similar to traps. These events are potentially of
TOPS-20 Dynamic Library Requirements Page 7
SOFTWARE CAPABILITIES
global interest, so the ability to detect them should be considered a
resource, and this resource must be made sharable.
On the other hand, receipt of a decnet or IPCF message is not an event
of general interest. The ability to detect these message is a resource,
but not one that needs to be sharable since only the package waiting for
the message really cares. However, the PSI channels used to capture
such events are resources, and must be allocated centrally to avoid
conflicts.
For the purposes of this document, any of the following will be referred
to as the occurrence of a "condition": occurrence of an APR trap, an
interrupt on a fixed channel, an interrupt on a channel assigned to
character interrupts. Possibly, conditions could also be initiated by
software, perhaps by a call to a DYNLIB routine. This is very similar
to the VMS concept of condition.
For a routine in a program (or library) to "handle" a condition means
for that routine to be informed of that condition, and take whatever
action is appropriate and possible.
For DYNLIB to "handle" a condition means for DYNLIB to route the
condition to appropriate user-supplied routines for handling. Every
routine in the chain of calls from the top-level program to the routine
within which the condition occurred should have the right to specify
("enable") a condition handling routine. We refer to this as
"hierarchical handling" of the condition because the condition is called
to the attention of a hierarchy of handling routines.
REQUIREMENT: DYNLIB must provide rules and facilities for hierarchical
handling of: APR traps, character interrupts, interrupts on PSI
channels with fixed meanings. These facilities must be available
anywhere within any program using DYNLIB, not just from within dynamic
libraries.
REQUIREMENT: A routine in a program using DYNLIB must have the option
of "enabling for condition handling" by specifying to DYNLIB a routine
which will handle any conditions occurring in the enabling routine or
beneath it which are not handled by handlers enabled lower down. The
handler routine enabled must then be called for each condition it is
eligible to handle. When the routine that enabled the handler exits,
the handler must be disabled.
RESTRICTION: It is allowable to require that routines which enable a
handler must make a DYNLIB call to disable the handler before they exit.
REQUIREMENT: A condition handler routine must have at least the
following options when called for a condition: pass the same condition
on to the next higher handler, ignore the condition and continue from
the point where the condition occurred, and unwind the stack causing a
return from the handler's establisher with a return value specified by
the handler.
TOPS-20 Dynamic Library Requirements Page 8
SOFTWARE CAPABILITIES
RESTRICTION: It is acceptable to require routines which enable a
condition handler to be called with some restricted set of linkages.
GOAL: Handling character interrupts should use as few channels as
possible.
REQUIREMENT: DYNLIB must provide rules and facilities for allocating
the assignable PSI channels.
REQUIREMENT: DYNLIB must provide hierarchical handling of
software-originated conditions. This facility does not need to interact
with BLISS signalling. This facility must, however, coexist with BLISS
signalling in the same routines, modules, and programs.
As a grafted-on facility, it is not expected that DYNLIB condition
handling will perform as well as built-in facilities such as BLISS
condition handling or VMS condition handling. It is not expected that
DYNLIB condition handling will be the primary means of handling
exception conditions in most programs. However, these performance
problems appear unavoidable given the major goal of facilitating sharing
of unique resources such as the trap and PSI systems.
REQUIREMENT: Meeting the goals in this section must not impose any
additional overhead on routines not enabling for any of these
conditions.
GOAL: Routines enabling for these conditions should not incur more than
20 additional instructions and 10 words of additional data.
4.2.4 Error Handling
Although it is desirable that only a single piece of code be written to
supply any given service, the interface presented to the user of a
program must be controllable by the top-level program. In particular,
packages called as subroutines shouldn't do any terminal I/O unless
specifically requested to by the top level.
REQUIREMENT: DYNLIB must provide rules and/or facilities allowing
packages called through DYNLIB to report errors back to their callers
without loss of important information.
4.2.5 Initialization
With TOPS-20 release 6, DIGITAL is shipping the multi-forking EXEC as
the standard version to the field. Since many products take significant
amounts of time to initialize (due to finding and mapping in dynamic
libraries, processing initialization files, opening log files, or
whatever), many users will probably form the habit of keeping around the
forks which contain their more commonly used utilities. This makes it
more important than ever that utilities be written to be restartable.
TOPS-20 Dynamic Library Requirements Page 9
SOFTWARE CAPABILITIES
REQUIREMENT: Instructions must be provided on how to write dynamic
libraries and main programs so that the resulting programs are
restartable. It is not acceptable to require that dynamic libraries be
found and mapped in again on a restart.
4.2.6 Debugging
REQUIREMENT: There must be good support for debugging dynamic
libraries. The standard debugger used for debugging programs written in
any language must work for debugging dynamic libraries written in that
language before we can claim that dynamic libraries can be written in
that language.
RESTRICTION: It is acceptable to require that BLISS-36 programs be
debugged using DDT rather than SIX12.
4.3 De-linking
Earlier work on monitor support for dynamically linked libraries has
specified special monitor actions when saving a memory image of a
program which used dynamic libraries. Some of those actions are not
compatible with the requirements for DYNLIB.
EXTERNAL REQUIREMENT: it must be possible to save a memory image of a
program that called dynamic libraries that includes all the libraries
that it had mapped in up to the time of the save, and their internal
states excepting AC's and open files.
RESTRICTION: it need not be possible to save a program, after running
it, in such a way that running the saved image will call in new copies
of any dynamic libraries used.
4.4 "Streaming"
"Single-stream" libraries are those like the current SORT which can only
handle one stream of operations at a time, but which have a user
interface such that you make more than one call for that single stream
(some sort of state information is preserved between calls).
"Non-streamed" libraries are those like, perhaps, the MTHLIB, which can
only handle one stream of operations, but where this doesn't matter to
anybody because there is only one call made to perform the operation.
There is no "state" saved across calls to these libraries. Note that to
really qualify in this category a library must be fully reentrant.
TOPS-20 Dynamic Library Requirements Page 10
SOFTWARE CAPABILITIES
"Multi-streamed" libraries are those like callable Datatrieve which
support several concurrent streams of operations. This classification
can be further divided into "infinitely multi-streamed," where it should
always be possible to start another stream, and "limited
multi-streamed," where you could easily run out of streams. Where
"multi-streamed" is used without qualification, "infinitely
multi-streamed" should be assumed.
REQUIREMENT: DYNLIB must work with non-streamed and infinitely
multi-streamed libraries.
4.5 Library Version Checking
When a library is called using DYNLIB, the association between the call
and the particular version of the library obtained is made at run-time.
Libraries must be allowed to evolve and improve, which will result in
some cases in changes to the addresses exported from a library. It is
necessary to provide some way to check whether the version of a library
actually found is compatible with the version the caller was built to
call.
REQUIREMENT: Dynamic libraries must have version numbers associated
with them. These numbers must be stored as part of the library. A
program which calls dynamic libraries must contain the version of each
library it calls that it was built for.
REQUIREMENT: When DYNLIB finds and maps in a dynamic library, it must
compare the version number of the library against the version number
expected by the caller and return an error if the version found is not
acceptable. Which versions are acceptable must be definable by the
library or the caller.
4.6 DYNLIB Version Checking
To allow for the possibility of major improvements to DYNLIB, DYNLIB
should be assigned a version number; the version should be checked
whenever DYNLIB is invoked.
REQUIREMENT: DYNLIB must be assigned a version number. DYNLIB must
know at run-time what version number it is.
REQUIREMENT: Callers and dynamic libraries should contain the number of
the version of DYNLIB they are built for. When DYNLIB performs a
service for a caller, it should first check if the version of DYNLIB
actually in use is acceptable to the caller and, if one is involved, the
dynamic library. Acceptability of versions of DYNLIB should be defined
in DYNLIB. If the version of DYNLIB in use is not acceptable to the
caller or to the callee, the DYNLIB function requested must return an
error indication as described below.
TOPS-20 Dynamic Library Requirements Page 11
PUBLICATIONS
5.0 PUBLICATIONS
REQUIREMENT: There must be documentation aimed at users of dynamic
libraries.
REQUIREMENT: There must be documentation aimed at writers of dynamic
libraries.
There will probably be many more people calling dynamic libraries than
there will be writing dynamic libraries.
GOAL: The "user" and "writer" documentation should be separate.
GOAL: The DYNLIB documentation should be published manuals.
REQUIREMENT: For any languages in which we wish to claim that it is
possible to write dynamic libraries, there must be language-specific
documentation on how to do so. Such documentation must be produced for
at least MACRO and BLISS.
GOAL: For at least the major higher-level languages (FORTRAN, COBOL,
PASCAL), there should be documentation on how to call dynamic libraries
from the language.
RESTRICTION: Since DYNLIB is not a utility directly used by users,
there need not be any HELP files associated with DYNLIB.
6.0 PACKAGING
The DYNLIB facility is being developed for Datatrieve. We want to make
it available to all layered products in the longer run.
REQUIREMENT: There must be a plan to ship all DYNLIB pieces necessary
to write, debug, and use dynamic libraries to all TOPS-20 customers.
RESTRICTION: DYNLIB may ship with individual products that need it
before shipment to all sites is achieved.
7.0 INSTALLABILITY
DYNLIB may ship either alone or with other products. If DYNLIB ships
with other products, then installability is the responsibility of those
other products' teams.
REQUIREMENT: DYNLIB shipped alone must include an installation
verification procedure.
TOPS-20 Dynamic Library Requirements Page 12
INSTALLABILITY
REQUIREMENT: DYNLIB shipped alone must be installable in under 1 hour
including unpacking, reading instructions, performing the installation,
and performing the installation verification procedure.
8.0 EASE OF USE
The requirements in this section also appear elsewhere in this document,
they are collected here for convenience.
REQUIREMENT: Using an address exported from a dynamic library must be
as similar as possible to using an address exported from a REL library.
REQUIREMENT: Invoking a dynamic library must be made as transparent as
possible (invoking the dynamic library should look as much as possible
like invoking a routine from a REL library). No special subroutine call
may be required to load a dynamic library. The library must be loaded
as a side-effect of referring to an address exported from it.
9.0 PERFORMANCE
Since DYNLIB is intended to be widely used as the basis for building
products out of layers of building-blocks, performance can be an
important issue. We believe, based on the usage of existing libraries,
that the vast majority of the interactions between a caller and a
package will be routine calls.
REQUIREMENT: Calling a routine in a dynamic library must not normally
require more than 3 machine instructions (not considering any page-fault
handling).
REQUIREMENT: Referring to an exported address in some manner other than
a routine call must not take longer or be harder to code than making an
indirect reference through an in-section pointer to an out-of-section
location.
RESTRICTION: Performance of the first reference to any address exported
from a library may be much slower, up to several seconds on a loaded
system.
RESTRICTION: Calling a dynamic library from section 0 is not subject to
the above performance constraints. The capability of calling a dynamic
library from section 0 is provided only as an aid to incremental
conversion of existing software.
The other DYNLIB functions are very infrequently used and are not
performance-critical.
TOPS-20 Dynamic Library Requirements Page 13
RELIABILITY
10.0 RELIABILITY
Because the first product using dynamic libraries to be seen by
customers, Datatrieve-20, is aimed at inexperienced users, it is
especially important that dynamic libraries not introduce any mysterious
(from the users' points of view) errors. Ideally, no errors should
occur. Next best is for errors to explain themselves clearly.
REQUIREMENT: DYNLIB will not ship with any known first or second
priority problems (the number of allowable lower priority problems will
be decided by the project team at that time). Problem priorities are as
defined for QAR's, and are included in an appendix of this document.
REQUIREMENT: No patches may be required to the shipped version before
use.
REQUIREMENT: In the event of being unable to satisfy a dynamic loading
request, DYNLIB must be capable of returning an error indication to the
caller if the caller is set up to handle it. The information made
available must include at least the following:
o Identification of the library it was trying to load, including
the file specification where it expected to find that library
o A description of the error
o If the error was a failing monitor call, the monitor error
message describing that failure
o The PC in the user's code of the call which invoked DYNLIB
o If applicable (if a DYNLIB bug seems possible), the PC in
DYNLIB at which the failure took place
DYNLIB use rule: It is recommended that all programs which call dynamic
libraries be written to accept this error information and present it to
the user in a manner compatible with the other user interactions made by
the program.
If a program chooses not to take advantage of this facility, DYNLIB
should present the error information directly to the user. This
violates one of the prime rules of layering (it does terminal IO other
than at the request of the top level), which can result in (for example)
screen formatting errors, but we consider this better than not reporting
the error at all.
REQUIREMENT: If no provision has been made by the calling program to
handle a DYNLIB error, DYNLIB must print an error message on the
terminal which is as clear and descriptive as possible. It must include
at least the information listed as required for the error return above.
We will consider DYNLIB to be adequately reliable if, during the first
12 months after FRS, unique error rates average less than 1 per month
total in priorities 1 and 2, less than 3 per month in priorities 3-5.
Problem report rates may be up to 5 times this (i.e. an average of 5
submissions of each problem).
TOPS-20 Dynamic Library Requirements Page 14
MAINTAINABILITY
11.0 MAINTAINABILITY
DYNLIB will be designed and written with ease of maintainability in
mind.
REQUIREMENT: Maintainability must be taken into account when choosing a
language in which to implement DYNLIB.
RESTRICTION: Because of the high level of interaction probably required
between DYNLIB and the monitor and hardware, portions or all of DYNLIB
may be written in MACRO.
REQUIREMENT: Any software necessary for defining libraries must be
normally available at all supported TOPS-20 sites. In particular, BLISS
must not be necessary to define a library.
REQUIREMENT: All project documents must be formally inspected and the
problems found corrected.
GOAL: As much as possible of DYNLIB will be housed in a dynamic
library. This makes it possible to update DYNLIB itself without
requiring users to re-link all programs using DYNLIB.
RESTRICTION: Due to the time constraints on this project, it is not
necessary to produce an additional document describing the internals of
DYNLIB. However, the functional and design specs must be updated to
reflect the status of the code.
REQUIREMENT: DYNLIB will be autopatchable in the field.
REQUIREMENT: DYNLIB must not prevent dynamic libraries from being
autopatchable in the field.
SUCCESS CRITERION: The patch/update rate for the first 12 months will
be not more than 120% of the unique problem rate during that period.
12.0 MAINTENANCE
Maintenance of this product will be performed by Software Engineering.
REQUIREMENT: Updates must appear as necessary on the regular autopatch
tapes.
13.0 COMPATIBILITY
TOPS-20 Dynamic Library Requirements Page 15
COMPATIBILITY
13.1 Compatibility With Existing Libraries
There are a lot of existing REL libraries or other kinds of packages of
code. Many of them would be very useful as dynamic libraries --
consider SORT, DBCS, RMS, MTHLIB.
REQUIREMENT: It must be possible to take an existing REL library that
does not require trapping or interrupts and turn it into a dynamic
library with minimal changes to existing programs, in whatever language,
that call routines in the library.
RESTRICTION: Conversion of an existing REL library or other kind of
package may require considerable effort since dynamic libraries are
required to run in non-zero sections. Most existing packages do not
have this capability, but using the additional address space available
through extended addressing is basic to the goals of DYNLIB.
REQUIREMENT: Any systematic changes required in converting existing
packages into dynamic libraries must be documented in the manual on
writing dynamic libraries.
REQUIREMENT: There must be some tool in DYNLIB for interfacing programs
running in section zero to dynamic libraries called into non-zero
sections (note that most existing programs run in section zero).
13.2 Product Compatibility
Dynamic libraries are not particularly compatible with any previous
method of calling libraries (either linking in section zero, or GET% at
runtime into fixed addresses in section 0).
GOAL: Major languages should consider invoking their OTS routines as a
dynamic library. This would be a large step towards making mixed
language programming work. (If the OTS libraries were made to conform
to the interrupt and trap rules, a mixed language environment would be
achieved.) This would make it possible for dynamic libraries to be
written in higher level languages and called from other higher level
languages -- for example, a dynamic library of graphics routines could
be written in FORTRAN.
13.2.1 Dependency Issues
We must come to a consensus with the TOPS-20 monitor group on
interpretation of PDV's.
We must come to a consensus with the LINK and monitor groups on
LINK-provided memory maps.
TOPS-20 Dynamic Library Requirements Page 16
COMPATIBILITY
We must come to a consensus with the DDT, LINK, and monitor groups on
symbol table pointers in PDV's and elsewhere.
13.3 Standards Conformance
No known relevant standards.
13.4 Internationalization
There are no requirements in this area. This product interacts with the
end-user only in "last-ditch" error messages which should never be seen
from a program designed according to our recommendations. There is
therefore no need for foreign language versions of the product.
There is no benefit to making DYNLIB documentation available in foreign
languages unless policy is changed and all LCG documentation starts
becoming available in foreign languages. This is not anticipated.
REQUIREMENT: DYNLIB must not preclude use of 8-bit ASCII codes in
dynamic libraries or programs which call them.
14.0 EVOLVABILITY
REQUIREMENT: user-mode DYNLIB, if implemented, must be designed with an
eye to eventual conversion to monitor-mode DYNLIB. The maximum amount
of effort for converting programs and dynamic libraries from user-mode
DYNLIB to monitor-mode DYNLIB must not exceed: recompile the library
definitions using the new version of DYNLIB, and relink any programs
calling the libraries.
Customers should not be allowed to change any of the basic data
structures; however, they may need to store information relating to the
libraries which naturally seems to belong in our data structures.
Similarly, individual libraries may have special data storage
requirements that could best be served this way.
REQUIREMENT: At least a word in each DYNLIB data structure that resides
in program address space must be set aside for the use of customers. A
word must also be set aside for the use of the specific library that
this data structure relates to, if it relates to a specific library.
TOPS-20 Dynamic Library Requirements Page 17
COSTS
15.0 COSTS
DYNLIB is being developed as part of the Datatrieve-20 project.
We hope to limit the development of DYNLIB to 4 man-months (including
all time spent from writing specification to field-test entry).
16.0 TIMELINESS
The product must be ready to field-test and ship along with
Datatrieve-20.
17.0 CONSTRAINTS AND TRADES-OFF
Ease of conversion of an existing library is more important than keeping
the call overhead of that library within the bounds specified. However,
new libraries implemented according to the instructions to be developed
as part of DYNLIB must meet the call overhead limits. There may be
different techniques for defining a new dynamic library and a dynamic
interface to an existing library.
18.0 APPROVAL PROCESS
DYNLIB is being developed within the Datatrieve-20 project, but will
probably be used by many other projects. The normal approval process as
applied to Datatrieve-20 probably will not give sufficient visibility to
DYNLIB to ensure that necessary feedback from other groups is received.
Therefore, special efforts will be made to circulate DYNLIB documents to
project leaders and supervisors throughout LSG Software Engineering, and
some appropriate consultant-level people within the group will be
approached individually for their comments.
APPENDIX A
PROBLEM PRIORITIES
(From the QAR form instructions)
Refer to the following code in establishing a priority for your problem:
1. Most production work cannot be run; e.g. functions/jobs which
are not usable and are a major use of the system, such as
system will not BOOT, necessary peripherals are out of service.
2. Some production work cannot be run; e.g. certain
functions/jobs are not usable, performance degradation,
installation has insufficient excess capacity.
3. All production work can be run with some impact on user; e.g.
significant manual intervention required, extra procedures,
performance degradation, but installation has excess capacity.
4. All production work can be run with no significant impact on
users; e.g. problems can be easily patched, simple bypass
procedure exists.
5. No system modifications needed to return to normal production;
e.g. suggestion, consultation, documentation error.