.RIGHT MARGIN 67
.CENTER
DATATRIEVE-11 to VAX-DATATRIEVE Conversion Panel
.BLANK 2.CENTER
Joe H.#Gallagher
.BLANK.CENTER
Research Medical Center
.BLANK.CENTER
Kansas City, MO 64132
.BLANK 2.CENTER
Bart#Z.#Lederman
.BLANK.CENTER
Greenberg Bros.#Part.
.BLANK.CENTER
New York, N.Y. 10010
.BLANK 2.CENTER
Session Chairman:
.BLANK.CENTER
Alex L.#Lamb Jr.
.BLANK.CENTER
USASATCOMA
.BLANK.CENTER
AMCPM-SC-4G
.BLANK.CENTER
Ft.#Monmouth, N.J.
.BLANK 2.CENTER
Transcribed by B.#Z.#Lederman
.BLANK
.PARAGRAPH
This session was originally planned to be a recounting of users'
experiences in converting to VAX-DTR from DTR-11. Due to various
difficulties which arose between scheduling the session and the
symposium (such as non-functioning equipment, and jobs being
eliminated), the final panel had to speak partially from experience
and partially on generic terms. Nevertheless, many important points
were covered, and the information was judged to be quite useful by
those attending the session. This is not intended to be an exact
transcription of the session: rather, it simply presents the
information in a readable form. I will not attempt do distinguish
which person made what contributions. I would also like to
acknowledge the presence of Suellen Harris and Bill Opalka from
DEC, who sat in the audience and kept us from going astray. 
.BLANK 2.CENTER
Major Points.
.BLANK.CENTER
DTR-11 is an almost perfect subset of VAX-DTR.
.PARAGRAPH
DTR-11 is a nearly perfect subset of VAX-DTR. (It was, of course,
developed first, and VAX-DTR was developed later using the same
syntax so users could migrate.)# The primary point of difficulty
has to do with a difference in how the hardware addresses memory.
The PDP-11 cannot address a word which starts on an odd byte
boundary, while the VAX can: therefore, such data types as REAL,
DOUBLE, INTEGER, DATE, and other word, double word, and quad word
variables, must be aligned to word boundaries on the PDP-11. DTR-11
will automatically do this by inserting invisible bytes where
needed. Consider the following record definition:
.BLANK.NO JUSTIFY.NO FILL.TEST PAGE 5
01 RECORD.
    10 FIELD1 PIC X.
    10 FIELD2 PIC 99 USAGE IS INTEGER.
;
.BLANK.JUSTIFY.FILL
FIELD1 is one byte long, and FIELD2 is two bytes long. In VAX-DTR,
this record would be 3 bytes long: on the PDP-11, it will be 4
bytes long, as an extra byte has to be inserted between the two
fields to make FIELD2 start on an even byte boundary. DTR-11 will
do this wherever necessary (and only where necessary), and does not
issue warnings or error messages. This will become apparent if the
data files are moved from one system to another, and the same
record definition is used on the VAX without taking precautions. It
is possible to make VAX-DTR allocate on word boundaries by using
the clause
.BLANK
"ALLOCATION#IS#LEFT-RIGHT"
.BLANK
within the record definition; therefore, you should change the
record definition to:
.BLANK.NO JUSTIFY.NO FILL.TEST PAGE 8
DEFINE RECORD TEST__REC
USING
ALLOCATION IS LEFT-RIGHT
01 RECORD.
    10 FIELD1 PIC X.
    10 FIELD2 PIC 99 USAGE IS INTEGER.
;
.BLANK.JUSTIFY.FILL
If you do this, VAX-DTR will allocate space in the same way as
DTR-11, and there should be no problems. If you don't, you may get
an error message similar to "RECORD LENGTH DOES NOT MATCH" when you
try to READY the new domain on the VAX, which is an indication that
you may have an alignment problem. See also the section on moving
data below.
.PARAGRAPH
If you do get these messages, and don't do anything about it before
storing new data, you will corrupt the data file. It would be a
good precaution to be certain that the first time you READY the
domain after moving the data that you do so READ#ONLY, and look at
the data. If there are any alignment problems, you will immediately
see that data isn't coming out correctly, and can take corrective
measures.
.BLANK 2.CENTER
Planning the move: more dictionary options.
.PARAGRAPH
The Common Data Dictionary has more functionality than the
Dictionaries on the PDP-11, and some thought should be given to
using this. On the PDP-11, there is often a separate dictionary for
each application, and the data files often aren't shared. On the
VAX, all definitions go into the CDD, and makes sharing easier. The
CDD has a hierarchical structure, and if you have many separate
projects you will want to plan which sub-directories should hold
what definitions, and which definitions should go into a site
common directory (for everyone to use), which should go into
project wide directories, and which may be left in individual
directories. The protection options are also greater on the VAX
than on the PDP-11: if you are not using protection now, then you
can move with no protection without making any changes. If,
however, you have been using UIC type protection on the PDP-11,
then you may want to change, as it is unlikely that you will keep
the same UIC scheme on your VAX disks that you had on your PDP-11
disks. (VMS does have UICs, but most people go to named directories
and hierarchical sub-directories, as these tend to be easier to use
and more closely follow the way most data is organized.) Therefore,
you may want to change UIC and/or password protection, and use some
of the new protection options available in VMS.
.PARAGRAPH
You do not have to learn much about the CDD before moving if you do
everything in DTR. You can put everything into one dictionary like
DTR-11, but you will get better performance if you make a good
division from the beginning. You can move definitions from one
sub-section to another later, but it's best to make some plans
first. A quick read-through of the manual to understand the
concepts, or attending some DECUS sessions are good ideas. For the
new user, no new training is needed to begin with, as they will see
a dictionary just like what they are used to (with the addition of
version numbers), and can see one big dictionary or their own
dictionary, whichever matches the PDP-11 installation; but the
person planning the move and organizing the application should know
something about the CDD. The CDD structure can copy the existing
PDP-11 setup, but usually matches the company or application
organization. Arranging the dictionary well has both performance
and maintenance advantages. Note also that VAX-DTR can have startup
files for individual users just as DTR-11 has, to put individuals
into the proper dictionary and perform READYs or start procedures,
as needed: also, the dictionary that the user sees for PLOTS can be
different from that for all other dictionary elements, so the users
can get all of their domains, records, procedures, and tables from
an individual dictionary, and at the same time get the PLOTS from
the system wide dictionary. You might also give them a procedure to
set their dictionary into the system wide DEMO library if you want
them to play with YACHTS, EMPLOYEES, or other sample files supplied
with DTR. There is also a command procedure NEWUSER.COM that comes
with DTR that can be used to easily set up a dictionary for a new
user. In addition, there is also a logical name that designates the
CDD dictionary in which the user will start.
.BLANK 2.CENTER
Moving the data.
.PARAGRAPH
One must move both the data files, and the dictionary objects
(record definitions, domain definitions, tables, etc.). You can
EXTRACT individual definitions, but DTR-11 is supplied with a
utility called QXTR, which will extract all of the definitions in a
dictionary at one time into a single command file in a manner
similar to EXTRACT#ALL in VAX-DTR. This utility inserts the
"ALLOCATION#IS#LEFT-RIGHT" clause into the record definitions,
which should simplify the move considerably. [This is probably true
only for DTR-11 V3.0 and V3.1, and not for earlier versions of DTR.
QXTR is not supplied with PRO-DTR. DTR-11 itself does not insert
the ALLOCATION clause when extracting.]# If you are using
individual dictionaries for different users, you must extract each
dictionary individually. QXTR also gives you the option of
extracting the protection qualifiers, or not, as you choose. This
single file can then be moved to the VAX and invoked to re-create
all of your records, tables, domains, etc. One factor not covered
in the original talk is that explicit references to disks and/or
directories in domain definitions will probably have to be changed,
as your device names will change, and you will probably be using
symbolic device names and/or named directories, as mentioned
before. To actually get the information across, there are several
options.
.BLANK.CENTER
Moving data on disks.
.PARAGRAPH
If you are using RSX (which in this paper includes 11M, 11M-Plus,
IAS, and possibly 11D), and you have the same kind of disk drive on
both systems, or are going to move the disk drive to the new
system, then you can read your old disks on VMS. You will want to
copy the information to a new disk, to take advantage of some VMS
features like named directories, but the files can be read while on
the old disk, or copied. This applies both to the data files, and
the file created by QXTR (or the DATATRIEVE EXTRACT command). If
you are running RSTS/E, then you are out of luck: no other
operating system will read your disks. If you don't have a disk
type in common on both systems and are not moving the devices, you
might want to consider plugging in your old disk on the VAX just
long enough to copy the data, if possible. If not, then the local
DEC office may be able to copy the disks, or your local DECUS LUG
may help you find a user who may let you do it, or you may be able
to locate a commercial service company that will do it.
.BLANK.CENTER
Moving data by Network.
.PARAGRAPH
If both systems run DECNet (no matter what the operating system),
then any file that DATATRIEVE can read can be read over the
network. You can COPY (or NFT or FTS) the file with the definitions
to the VAX, define your domains, copy the data file, and be ready
to go. Alternatively, you can define a two domains with the same
record definition, with one having a normal file specification on
the VAX, the second having a file specification which includes the
node name for the PDP-11 (something you can do with VAX-DTR but not
DTR-11); or what may be better, if you have remote DATATRIEVE
installed on the PDP-11 you can use the "READY domain AT node"
feature of VAX-DTR, and simply read from the old domain on the
PDP-11 to the new domain on the VAX. This will be a little slow, as
DTR is not optimized for this type of operation the way DECNet
utilities are, and the data will be going over the network, but
hopefully it will only be done once. The advantage of doing this is
that it can avoid the problems of record alignment mentioned
before. By using the DTR-11 remote server on the PDP-11, the data
will be read exactly as it has always been read: on the VAX side,
you can define a record without the ALIGNMENT clause, and pack the
data in without worrying about hidden bytes. You may also want to
consider other methods of converting data mentioned below. Remote
DATATRIEVE is available starting with V3.0, not in earlier
versions. If you are not using DECNet, you may not have special
communications devices normally used for networking. DECNet will
work over normal asynchronous lines used for terminals, however.
You may have to give up 8 or 16 lines for a while, as DECNet grabs
the entire device on PDP-11s, and the maximum speed may be 9600
baud so transfers may be a little slow, but hopefully you will only
do the conversion once. If you don't have DECNet, maybe you can
persuade the local office to let you use it just for a few days
while you move your data.
.BLANK.CENTER
Moving data on Magnetic Tape.
.PARAGRAPH
If both systems have magnetic tape (or you are moving the tape
drive), you may be able to store the definitions and information on
tape. Once again, the RSX family provides the best compatibility,
with most systems writing ANSI tapes which can be read by VMS. (On
earlier RSX systems, this was a SYSGEN option, so check carefully
before you write out the tapes and disconnect your PDP-11.)# If
not, you can write DOS-11 format tapes on all RSX family systems
which can be read on the VAX with FLX (in compatibility mode) or
the new EXCHANGE utility. Though you might possibly get indexed
files over if transferred in image mode it will be much safer if
you first convert the data to a sequential file and re-construct it
on the VAX, as will be mentioned again later. If you have RSTS/E,
then you are again stuck with a system which works differently than
everyone else. You might be able to generate DOS-11 format tapes,
or you may want to upgrade to V9.0: this is the latest version of
RSTS/E, and it includes a utility which writes tapes that are
compatible with the VMS BACKUP utility. For RSTS/E users this will
probably be the easiest way to move files, though V9.0 is a new
release and those of us on the panel have not yet had any feedback
from users. Moving data from RSTS/E to other systems can be so much
of a problem that some people plan to upgrade to RSTS/E primarily
for the purpose of being able to use the new utility to write VMS
readable tapes with the new utility.
.PARAGRAPH
There is an RMS utility set, BCK and RST, which is intended for
backing up files to and restoring files from magnetic tape. These
utilities automatically convert indexed files to a form which will
store properly on magnetic tape, and place error checking and other
useful information on the tape for all types of files. Because they
are supplied as part of RMS, they should be available on all PDP-11
systems (that can run DATATRIEVE). The problem is that VMS may not
have a utility corresponding to RST to get the data off: so unless
you have compatibility mode on your VMS system (and it is now an
optional layered product), this may not be a viable option.
.BLANK.CENTER
Other file transfer methods.
.PARAGRAPH
If none of the above are available, you may want to look into some
of the communications packages which will operate over normal
serial (terminal) lines. KERMIT is a public domain program
available from the DECUS library (and elsewhere) which runs on
virtually any computer and operating system, will work on serial
lines, does error checking, and can transfer (in most cases) both
text and binary files. It may be slow, but it will work. There are
also other communications packages that perform similarly: you may
even be able to use SET#HOST/DTE/LOG on the VAX to go in to the
PDP-11 and type the file out (this works best if the data is all
text) or a similar "dumb" text transfer.
.BLANK 2.CENTER
Re-Organizing your data.
.PARAGRAPH
VMS has some options not available on the PDP-11 for indexed files,
notably Prolog-3, which allows some space saving. Since indexed
files should be re-organized occasionally for best performance, and
since the same data will occupy less space in a sequential file and
take less time to transfer over networks, and transfer more easily
on magnetic tape, the time of transfer would also be a good
opportunity to re-organize and optimize the data file by converting
it to sequential, and re-populating an indexed file on the VAX.
Even if you can move your disk packs and can read your old files
directly, re-organizing at this point is a good idea, though in
this case you don't have to convert to an intermediate sequential
file.
.PARAGRAPH
First, you need a description of the current indexed file: you can
simply make a note of the record length (which you get from DTR
when you define the record), and figure out where the keyed fields
are in the record, but there are two RMS utilities, the older DSP
and the newer DES which are designed to record file characteristics
and define new files: it's a good idea to have such a description
file for documentation purposes, even if you aren't anticipating a
conversion. DES creates a file definition which is quite similar to
that used by the VMS FDL utility: in actual tests, I was very
surprised to find that FDL will actually read the descriptor file
produced by DES! It may give you a few warning messages, especially
if the SOURCE and TARGET fields are empty, but this shouldn't cause
any serious problems. If the utility refuses to read your
descriptor file and aborts, check to see that when you transferred
the description file over it did not pick up trailing blanks on the
description items, and that the file position qualifier says NONE
with no numbers trailing. You can use a regular text editor such as
EDT to go over the file before reading it with EDIT/FDL if
necessary to touch it up. If you don't take this approach, then you
can move the record definition, define a domain and define a file:
DTR will create a file that matches the record definition. You can
then look at that file with EDIT/FDL if you wish, or do the
optimization shown below. If you move your disk packs FDL can
probably read your original data file, and may also be able to do
this over the network if you are using DECNet.
.PARAGRAPH
Converting the data file from indexed to sequential on the PDP-11
is quite simple: the CNV utility will perform this conversion by
default. Simply give the name of the indexed file on input, and the
file name you want for the sequential file on output.
.PARAGRAPH
Once the file description and the data file are on the VAX, you can
populate the file with CONVERT: however, now would be a good time
to review the file design for possible performance improvements.
the FDL utility (EDIT/FDL) has an OPTIMIZE script which can make
the decisions a little easier, as it will look at your data file
and give you some information about what can be done to improve
access. The one factor which comes up quite often is bucket size:
on the PDP-11, bucket size is almost always the smallest possible
value that will hold the record size you are using, as larger
buckets use up pool space. On the VAX, this is no longer a problem,
and larger bucket sizes are a viable option. If you often retrieve
records which are next to each other, such as retrieving a record
by the primary key and then reading the next several records in
order, then a larger bucket size may improve performance. If,
however, you retrieve records scattered all over the file in no
particular order, then a larger bucket size won't help and may
hurt, but a different index structure may be of benefit. If you
don't know what options to take, use the FDL utility and let it
optimize: it will usually take reasonable options. Once this is
done, you can populate a new file again with CONVERT.
.BLANK 2.CENTER
Combining related domains.
.PARAGRAPH
Another consideration in DTR-11 is that the data is sometimes
separated into several domains rather than one, to prevent the
record definition and buffers from getting so large as to use up
all of your pool space. When converting to VAX-DTR, you may want to
recombine them by moving all of the individual files (and domain
definitions) over, define a VIEW to combine them and a single
record definition with all of the fields, and read from one into
the other. This may be a little slow, but again, this will only be
done once. You will then want to use FDL as described before to
optimize the new combined file.
.BLANK 2.CENTER
Reports.
.PARAGRAPH
For reports, the VAX-DTR works very much the same as DTR-11. If you
have reports which are adjusted so that field breaks occur just at
the beginning or end of a page, or are otherwise 'finagled' to
match a pre-printed form, you may find that you have to do a little
adjustment to the lines per page or number of lines skipped
qualifiers, but most straight forward reports will work with no
changes.
.BLANK 2.CENTER
Expanded features in VAX-DATATRIEVE.
.BLANK.CENTER
Tables.
.PARAGRAPH
While VAX-DTR has dictionary tables just like DTR-11, it also has
domain tables: this allows data in a domain to also be accessed
like a table. One field (preferably a keyed field for good
performance) takes the place of the "left" side of the table, and
another field (any one in the domain) takes the place of the table
entry on the "right". Dictionary tables are faster for small
tables: domain tables can be faster for large tables if keyed
fields are used, can take the place of several tables, and can also
be accessed as a regular domain, which makes changing the table the
same as modifying the data in any other domain; therefore, tables
which are modified often are easier to maintain as domain tables.
.BLANK.CENTER
Functions.
.PARAGRAPH
VAX-DTR has all of the functions (MIN, MAX, TOTAL, etc.) that
DTR-11 has, plus several more (standard deviation, and running
count, for example). In addition, there is a new set of functions
of the FN$xxx family which can do things like convert lower case
text to upper case, move sub-sections of text strings, format
output strings, get system information, do mathematical operations,
and many other functions. If that isn't enough, you can add your
own functions. In many cases, things that you are doing, perhaps
with some difficulty, with procedures or COMPUTED__BY clauses may
be much easier with the extra functions available in VAX-DTR, and a
review of what you are doing may reveal some areas where
improvements may be made. Though you may want to wait until your
application is moved over and running, this is an area where you
will want to do some work quite soon after the move
.BLANK 2.CENTER
^&No Pool Space Restrictions!\&
.PARAGRAPH
In DTR-11, procedures are often broken into small pieces to
conserve pool space, and more things were done in command files.
You may consider moving more of the work into DTR procedures,
especially when you can get large blocks into single WHILE or
BEGIN-END blocks. This may take a little longer to compile, but
once compiled will execute faster than separate blocks.
.PARAGRAPH
In DTR-11, especially if your application was just on the edge of
available pool space, things tended to be pared down to the
minimum, especially in record definitions. Variable names were very
short, edit-strings removed, etc. On the VAX, you have the
opportunity to put more descriptive field names back in, and make
the fields more self-documenting, put in query names, edit strings,
and so on. Don't drop your good habits, however. Some users think
they have infinite space on the VAX, and don't FINISH unused
domains or RELEASE unused variables. This will cost you something
eventually on the VAX (usually in paging), and it won't be as
obvious as running out of pool space was in DTR-11: your
application, or the whole system, just slowly degrades. You also
may interfere with other users who want to access the data if you
are keeping files open and are possibly locking records. You can
keep domains open if you expect to use them again and save the time
it would take to READY the domain, but when you are done with a
domain you should FINISH it.
.BLANK 2.CENTER
Other improvements.
.PARAGRAPH
All users will be happy to learn that VAX-DTR uses EDT when
editing, rather than the built-in editor of DTR-11. If you have an
EDTINI.EDT command file in your account, it will be used when you
edit something in VAX-DTR; keypad and all other commands will work,
etc. You will also notice that with the newer versions of VAX-DTR,
you have version numbers on definitions, so you can keep the old
definition around until you find that the new version works. Users
should be reminded to purge out old versions regularly.
.PARAGRAPH
An option available in VAX-DTR is the use of FMS or TDMS to have
form driven screens. After your data is on the VAX and working, you
may want to start thinking of converting some of your old
procedures, especially if they were working like menus, to form
driven screens. Something you do need to consider before you do
this, however, is whether to buy FMS or TDMS: they look very much
the same to DTR, so the choice is often determined by what other
software you are using. Some packages will require one or the
other, (for example, All-In-One uses FMS) while DTR can work with
either, one at a time.
.PARAGRAPH
There are various other features that are present in VAX-DTR (three
types of concatenation rather than two, CROSS statements, the
CHOICE statement, and all of the graphics capabilities), which you
will soon discover and will want to incorporate into your
applications; but none are needed immediately for conversion, and
you can wait to learn about them until after the problems of
conversion are over.