SPORADIC DIALOGUES
by Eugene Volokh, VESOFT
Q&A column published in INTERACT Magazine, 1987-1989.
Published in "Thoughts & Discourses on HP3000 Software", 4th ed.
Q: I have what you might think to be a strange question -- where is
the Command Interpreter?
On my MPE/XL machine, I see a program file called CI.PUB.SYS, so I
guess that that's the Command Interpreter program (the one that
implements :RUN, :PURGE, etc.) -- however, I couldn't find such a
program on my MPE/V computer.
:EDITOR, I know, is implemented through EDITOR.PUB.SYS; :FCOPY is
FCOPY.PUB.SYS; all the compilers have their own program files. Where
is the CI kept?
A: This is a very interesting question -- one that bothered me for a
while several years ago. The answer is somewhat surprising.
Virtually all of the operating system on MPE/V is kept in the system
SL (SL.PUB.SYS). This includes the file system, the memory manager,
etc.; it also includes the Command Interpreter. But, you may say,
each job or session has a Command Interpreter process, and each
process must have a program file attached to it. Nope. Each of YOUR
processes must have a program file attached to it; MPE itself faces
no such restrictions.
All that a process really has to have is a data segment (which MPE
builds for each CI process when you sign on) and code segments;
internally, there's no reason why those code segments can't be in the
system SL. MPE just uses an internal CREATEPROCESS-like procedure
that creates a process given not a program file name but an address
(plabel) of a system SL procedure; no program file needed.
On MPE/XL, the situation is somewhat different (as you've noticed);
there's a program file called CI.PUB.SYS that is actually what is run
for each CI process. However, if you do a :LISTF CI.PUB.SYS,2,
you'll find that it's actually quite small (186 records on my MPE/XL
1.1 system); since Native Mode program files usually take a lot more
records than MPE/V program files, those 186 records can't really do a
lot of work. CI.PUB.SYS is really just a shell that outputs the
prompt, reads the input, and executes it by calling MPE/XL system
library procedures. MPE/XL library procedures (including the ones
that CI.PUB.SYS calls, either directly, or indirectly) are kept in
the files NL.PUB.SYS, XL.PUB.SYS, and SL.PUB.SYS.
Thus, in both MPE/V and MPE/XL, virtually all of the smarts behind
command execution is kept in the system library (or libraries).
Notable exceptions include all the "process-handling" commands, like
EDITOR, FCOPY, STORE/RESTORE (which use the program STORE.PUB.SYS),
and even PREP (which actually goes through a program called
SEGPROC.PUB.SYS). However, :FILE, :PURGE, :BUILD, and so on, are all
handled by SL, XL, or NL code.
Q: A couple of months ago I read in this column that you can speed up
access to databases by having somebody else open them first. Is
there some easy way that I can take advantage of this without having
to write a special program? I guess what I'd have to do is have a
job that DBOPENs the database and then suspends with the database
open, but how can this be easily done?
A: Indeed, if a TurboIMAGE database is already DBOPENed by somebody
then a subsequent DBOPEN will be faster than it would be if it were
the first DBOPEN. On my Micro XE, a DBOPEN of a database that nobody
else has open took about 2 seconds; a DBOPEN of a database that was
already opened by somebody else took only about 0.8 seconds. The
first DBGET/DBPUT/DBUPDATE against a dataset were also faster if the
database was already open (for the DBGET, about 50 milliseconds vs.
about 400-500 milliseconds).
How can you take advantage of this? Well, if the database is
constantly in use (e.g. it's opened by each of the many users that is
accessing your application system), then the time savings comes
automatically -- the first user will spend 2 seconds on the DBOPEN
and all subsequent users will spend 0.8 seconds, as long as the
database is open by at least one other user.
The problem arises when your database is frequently opened but is
left open for only a short time; for instance, if it is used by some
short program that only needs to do a few database transactions
before it terminates. Then, you might have hundreds of DBOPENs every
day, the great majority of which happen when the database is NOT
already open. You'd like to have one job hanging out there keeping
the database open so that all subsequent DBOPENs will find the
database already opened by that job.
How can this be done without writing a special custom program? (I
agree that you should try to avoid writing custom programs whenever
possible -- if you wrote a special program, you'd have to keep track
of three files [the job stream, the program, and the source] rather
than just the job stream.) Well, let's take this one step at a time.
Having a job stream that opens your database is easy -- just say
!JOB OPENBASE,...
!RUN QUERY.PUB.SYS
BASE=MYBASE
<<password>>
1 <<the mode>>
Unfortunately, as soon as this job opens the database, it'll start to
execute all the subsequent QUERY commands; eventually (because of an
>EXIT or an end-of-file) QUERY will terminate and the database will
get closed. It would be nice if QUERY had some sort of >PAUSE
command (or, to be precise, a >PAUSE FOREVER command), but it
doesn't.
Or does it? What are the mechanisms that MPE gives you for
suspending a process? Can we use any of them from within QUERY?
Well, there's obviously the PAUSE intrinsic, which pauses for a given
amount of time. It's not quite what we want (since we want to pause
indefinitely), but more importantly there is really no way of calling
PAUSE from within QUERY (unless your QUERY is hooked with VESOFT's
MPEX). So much for that idea. There are also a few other intrinsics
-- for instance, SUSPEND and PRINTOPREPLY -- that suspend a process,
but they're not callable from QUERY either. You can't call
intrinsics from QUERY; you can't even do MPE commands (though in any
event there aren't any MPE commands that suspend a process).
One other thing, however, comes to mind -- message files. A read
against an empty message file is a good way to suspend a process;
however, QUERY doesn't have an ">FOPEN" or an ">FREAD" command,
either, does it?
Well, in a way it does. QUERY's >XEQ command lets you execute QUERY
commands from an external MPE file, and to do that it has to FOPEN it
and FREAD from it. You might therefore say:
!JOB OPENBASE,...
!BUILD TEMPMSG;MSG;TEMP;REC=,,,ASCII
!RUN QUERY.PUB.SYS
BASE=MYBASE
<<password>>
1 <<the mode>>
XEQ TEMPMSG
EXIT
!EOJ
The XEQ will start reading from this temporary message file, see that
it's empty, and hang, waiting for a record to be written to this
file. Since the file is a temporary file, nobody else will be able
to write to it, so the job will remain suspended until it's aborted.
There are a few other alternatives along the same lines. For one,
you could make the message file permanent -- that way, you can stop
the job by just writing a record to this file. Why would you want do
this? Because you may want to have your backup job automatically
abort the OPENBASE job so that the database can get backed up.
If the message file were permanent, your backup job could then just
say:
!FCOPY FROM;TO=OPENMSG.JOB.SYS
<< just a blank line -- any text will do >>
...
The one thing that you'd have to watch out for is the security on the
OPENMSG.JOB.SYS file -- you wouldn't want to have just anybody be
able to write to this file because any commands that are written to
the OPENMSG file will be executed by the QUERY in the OPENBASE job
stream. (Remember the >XEQ command.)
If you don't use the permanent message file approach, you can still
have the background job abort the OPENBASE job by saying
!ABORTJOB LOCKBASE,user.account
However, for this, you'd either have to have :JOBSECURITY LOW and
have your backup job log on with SM capability, or have the :ABORTJOB
command be globally allowed (unless you have the contributed ALLOWME
program or SECURITY/3000's $ALLOW feature).
Another alternative is to >XEQ a file that requires a :REPLY rather
than a message file, e.g.
!JOB OPENBASE,...
!FILE NOREPLY;DEV=TAPE
!RUN QUERY.PUB.SYS
BASE=MYBASE
<<password>>
1 <<the mode>>
XEQ *NOREPLY
EXIT
!EOJ
The trouble with this solution is that the console operator might
then accidentally :REPLY to the "NOREPLY" request. (Also, the job
wouldn't quite work if you had an auto-:REPLY tape drive!)
In any event, though, one of the above solutions might be the thing
for you. It's not going to buy you too much time (it only saves time
for DBOPENs and the first DBGETs/DBPUTs/DBUPDATEs on each dataset),
it isn't necessary if the database is already likely to be opened all
the time, and it won't work on pre-TurboIMAGE or MPE/XL systems.
However, in some cases it could be a non-trivial (and cheap!)
performance improvement.
One note to keep in mind (if you really look carefully at the way
IMAGE behaves): the BASE= in the job stream will only open the
database root file -- the individual datasets will remain closed.
HOWEVER, once the keep-the-database-open job starts, any open of any
dataset by any user will leave the dataset open for all the others;
even when the user closes the database, the datasets will still
remain open. If the user only reads the dataset (and doesn't write
to it), the dataset will only be opened for read access; however,
once any user tries to write to the dataset, the dataset will be
re-opened for both read and write access.
The upshot of this is that the database will start out with only the
root file opened, and then (as users start accessing datasets in it)
will slowly have more and more of its datasets opened. After each
dataset has been accessed (especially written to) at least once, no
more opens will be necessary until the last user of the database
(most likely the keep-open job itself) closes it.
Q: I recently tried to develop a way to periodically check my system
disk usage (free, used, and lost). I based it on information that
VESOFT once supplied in one of their articles.
My procedure was as follows:
1. Do a :REPORT XXXXXXXX.@ into a file to determine total disk
usage by account.
2. Subtotal the list (in my case, using QUIZ) to get a system disk
space usage total.
3. Run FREE5.PUB.SYS with ;STDLIST= redirected to a disk file.
4. Use QUIZ to combine the two output files and convert the totals
to Megabytes.
This way, I can show the total free space and total used space on one
report for easy examination. I can also add these two numbers, compare
the sum to the total disk capacity, and thus determine the 'lost' disk
space.
My question is in regard to the lost disk space. The first time I ran
the job, the total lost disk space came out to be approximately 19
Megabytes. After doing a Coolstart with 'Recover Lost disk Space', the
job again showed a total lost disk space of 19 Megabytes. Didn't the
Recover Lost disk Space save any space? Could there be something I'm
overlooking?
A: Unfortunately, there is. Paradoxical as it may seem, the sum of the
total used space and the total free space is NOT supposed to be the
same as the total disk space on the system; or, to be precise, there
is more "used space" on your system than just what the :REPORT command
shows you.
The :REPORT command shows you the total space occupied by PERMANENT
disk FILES. However, other objects on the system also use disk space:
- TEMPORARY FILES created by various jobs and sessions;
- "NEW" FILES, i.e. disk files that are opened by various processes
but not saved as either permanent or temporary;
- SPOOL FILES, both output and input (input spool files are
typically the $STDINs of jobs);
- THE SYSTEM DIRECTORY;
- VIRTUAL MEMORY;
- and various other objects, mostly very small ones (such as disk
labels, defective tracks, etc.).
Your job stream, for instance, certainly has a $STDLIST spool file and
a $STDIN file (both of which use disk space), and might use temporary
files (for instance, for the output of the :REPORT and FREE5); also,
if you weren't the only user on the system, any of the other jobs or
sessions might have had temporary files, new files, or spool files.
In other words, to get really precise "lost space" results, you ought
to:
* Change your job stream to take into account input and output
spool file space (available from the :SHOWIN JOB=@;SP and
:SHOWOUT JOB=@;SP commands). Since :SHOWIN and :SHOWOUT output
can not normally be redirected to a disk file, you should
accomplish this by running a program (say, FCOPY) with its
;STDLIST= redirected and then making the program do the :SHOWIN
and :SHOWOUT, e.g. by saying
:RUN FCOPY.PUB.SYS;INFO=":SHOWOUT JOB=@;SP";STDLIST=...
* Consider the space used by the system directory and by virtual
memory (these values are available using a :SYSDUMP $NULL).
* Consider the space used by your own job's temporary files.
* Run the job when you're the only user on the system.
Your best bet would probably be to run the job once, immediately after
a recover lost disk space, with nobody else on the system; the total
'unaccounted for' disk space it shows you (i.e. the total space minus
the :REPORT sum minus the free disk space) will be, by definition, the
amount of space need by the system and by your job stream. You can
call this post-recover-lost-disk-space value the 'baseline'
unaccounted-for total.
If your job stream ever shows you an 'unaccounted for' total that is
greater than the baseline, you'll know that there MAY be some disk
space lost. To be sure, you should
* Run the job when you're the only user on the system.
* Make sure that your session (the only one on the system) has no
temporary files or open new files while the job is running.
If you do all this, then comparing the 'unaccounted for' disk space
total against the baseline will tell you just how much space is really
lost.
Incidentally, note that you needn't rely on your guess as to the total
size of your disks -- even this can be found out exactly (though not
easily). The very end of the output of VINIT's ">PFSPACE ldev;ADDR"
command shows the total disk size as the "TOTAL VOLUME CAPACITY".
Thus, you could, in your job, :RUN PVINIT.PUB.SYS (the VINIT program
file) with ;STDLIST= redirected to a disk file and issue a ">PFSPACE
ldev;ADDR" command for each of your disk drives. (If you want to get
REALLY general-purpose, you can even avoid relying on the exact
logical device numbers of your disks by doing a :DSTAT ALL into a disk
file and then converting the output of this command into input to
VINIT).
Finally, it's important to remember that all this applies ONLY to
MPE/V. The rules of the game for MPE/XL are very, very different; in
any event, disk space on MPE/XL should (supposedly) never get lost.
Q: I want to "edit" a spool file -- make a few modifications to it
before printing it. SPOOK, of course, doesn't let you do this, so I
decided just to use SPOOK's >COPY command to copy the file to a disk
file, use EDITOR to edit it, and then print the disk file. However,
when I printed the file, the output pagination ended up a lot
different from the way it was in the original spool file! Is there
some special way in which I should print the file, or am I doing
something else wrong?
A: Well, first of all, there is a special way in which you must print
any file that used to have Carriage Control before you edited it with
EDITOR. When EDITOR /TEXTs in a CCTL file, it conveniently forgets
that the file had CCTL -- when it /KEEPs the file back, the file ends
up being a non-CCTL file (although the carriage control codes remain
as data in the first column of the file). To output the file as a CCTL
file, you should say
:FILE LP;DEV=LP
:FCOPY FROM=MYFILE;TO=*LP;CCTL
The ";CCTL" parameter tells FCOPY to interpret the first character of
each record as a carriage control code.
Actually, you probably already know this, since otherwise you would
have asked a slightly different question. You've done the :FCOPY
;CCTL, and you have still ended up with pagination that's different
from what you had to begin with. Why?
Unfortunately, not all the carriage control information from a spool
file gets copied to a disc file with the >COPY command. In particular:
* If your program sends carriage control codes to the printer using
FCONTROL-mode-1s instead of FWRITEs, these carriage control codes
will be lost with a >COPY.
* If the spool file you're copying is a job $STDLIST file, the
"FOPEN" records (which usually cause form-feeds every time a
program is run) will be lost.
* And, most importantly, if your program using "PRE-SPACING"
carriage control rather than the default "POST-SPACING" mode, the
>COPYd spool file will not reflect this.
Statistically speaking, it is this third point that usually bites
people.
The MPE default is that when you write a record with a particular
carriage control code, the data will be output first and then the
carriage control (form-feed, line-feed, no-skip, etc.) will be
performed -- this is called POST-SPACING. However, some languages
(such as COBOL or FORTRAN) tell the system to switch to PRE-SPACING
(i.e. to do the carriage control operation before outputting the
data), and it is precisely this command -- the switch-to-pre-spacing
command -- that is getting lost with a >COPY.
Thus, output that was intended to come out with pre-spacing will
instead be printed (after the >COPY) with post-spacing; needless to
say, the output will end up looking very different from what it was
supposed to look like.
What can you do about this? Well, I recommend that you take the disc
file generated by the >COPY and add one record at the very beginning
that contains only a single letter "A" in its first character. This
"A" is the carriage control code for switch-to-pre-spacing, and it is
the very code that (if I've diagnosed your problem correctly) was
dropped by the >COPY command. By re-inserting this "A" code, you
should be able to return the output to its original, intended format.
Now, this is only a guess -- I'm just suggesting that you try putting
in this "A" line and seeing if the result is any better than what you
had before. There might be other carriage control codes being lost by
the >COPY; there might be, for instance, carriage control transmitted
using FCONTROLs, or your program might even switch back and forth
between pre- and post-spacing mode (which is fairly unlikely).
However, the lost switch-to-pre-spacing is, I believe, the most common
cause of outputs mangled by the SPOOK >COPY command.
Robert M. Green of Robelle Consulting Ltd. once wrote an excellent
paper called "Secrets of Carriage Control" (published, among other
places, in The HP3000 Bible, available from PCI Press at
512-250-5518); it explains a lot of little-known facts about carriage
control files, and may help you write programs that don't cause
"interesting" behavior like the one you've experienced. Carriage
control is a tricky matter, and Bob Green's paper discusses it very
well.
Q: I sometimes have to restore files from my system backups, which are
usually about seven reels. If the file I want is, say, on the sixth
reel (but I don't know it), I have to mount each one of reels 1
through 6 and wait for :RESTORE to read through each one in its
entirety before it finds my file.
I know we ought to keep the full SYSDUMP listing so that we can
tell which file is on which reel, but that can be a pretty big
printout (12,000 files = 200 pages). Isn't there some way for :SYSDUMP
to put some sort of directory on the first reel that maps every file
to its reel number?
A: Well, unfortunately things aren't that simple (are they ever?).
Let's look at how :SYSDUMP (or :STORE) works.
Say that you say :STORE @.@.@. At first, :STORE has no idea how
many reels this operation will consume (who knows -- maybe you have a
10,000' reel!).
The first thing :STORE writes to tape is a "directory" containing
the names of all the files being stored. Note that so far :STORE
doesn't know which reel each file goes on, so this directory can't
contain this information.
Then, :STORE begins to write the files to tape. If you're lucky,
all the files will fit on one reel; if not, at some point :STORE will
get an "end of tape" signal from the tape hardware, and will know that
a new reel needs to be started.
Now, it would be great if :STORE could then go back to the
directory and at least mark each of the files that couldn't fit on
this reel. That way, when you mount the reel, :RESTORE can instantly
figure out if the file is on it or not.
Unfortunately, a tape drive is not a "read-write" medium. You can't
just go and, say, update the 10th record of a file that's stored on
tape. You can write a tape from scratch, or you can read it, but you
can't take an already-written tape and modify it.
Thus, we now have a reel that contains a directory with the names
of ALL the files in the fileset but only SOME of the actual files.
Only by reading all the way to the end of the reel can :RESTORE detect
that a particular file is actually on the next reel.
We're done with reel #1 -- now to reel #2. At the beginning of reel
#2, :STORE also writes out the same directory as it did on reel #1,
but also writes to the tape label THE NUMBER (relative to the start of
the fileset) OF THE FIRST FILE THAT IS ACTUALLY ON THIS TAPE. Then, it
writes out the files themselves, again hoping that all of them will
fit on this reel. If they don't fit, :STORE attempts to put them on
reel #3, reel #4, etc. In any case, the rule is that
EVERY REEL CONTAINS THE DIRECTORY OF ALL THE FILES IN THE ENTIRE
FILESET, PLUS THE NUMBER OF THE FIRST FILE ACTUALLY STORED ON THIS
REEL.
This, again, is because:
1. There's no way for :STORE to know in advance that any of the
files won't fit on the reel;
2. Once the reel is written, there's no way for :STORE to update
the directory on that reel short of re-writing the entire tape.
Thus, the bad news is that :STORE doesn't keep any table that tells
which reel each file is on. The good news is that it's still possible
to restore a file quickly even if you don't know its reel number --
JUST GO THROUGH THE REELS BACKWARDS!
That's right -- backwards. If you first mount reel #1, then reel
#2, then reel #3, etc., :RESTORE has to read each entire reel from
beginning to end, since only by reaching the end of tape marker can it
tell that the file is not on this reel. However, if you first mount
reel #7, then reel #6, then reel #5, etc., :RESTORE will only have go
through the DIRECTORY (a rather small chunk) of each reel to find the
reel that actually has the file.
In the example shown in the picture, say that you try to :RESTORE
file E. First you mount reel 3, whose directory indicates that it
contains files F and G (since F is the 6th file on the reel). Without
having to read through the entire reel, :RESTORE will realize that
file E is on some previous reel; it will then ask you to mount reel
#2, on which the file does indeed exist.
Of course, you might notice that :STORE *could* have put the reel
numbers INTO THE DIRECTORY OF THE LAST REEL, so that simply mounting
the last reel would automatically tell you exactly which reel number
the file you want is on. (By the time the last reel is generated,
STORE naturally knows onto which reels all of the other files were
written.)
Unfortunately, the authors of MPE didn't choose to do this, much to
our disadvantage. What most likely happened is that they didn't think
of it on the first version of the system, and when they realized
afterwards that this would have been a good thing, it was too late.
:STORE/:RESTORE faces a greater burden of compatibility than other
aspects of MPE -- not only do tapes generated by OLD versions of
:STORE need to be readable by NEW versions of :RESTORE, but tapes
generated by NEW versions of :STORE must also be readable by OLD
versions of :RESTORE. Once HP put out one version of the system with a
directory format that had no room for the reel number, it was stuck.
:FILE T;DEV=TAPE
:STORE A,B,C,D,E,F,G;*T
Reel #1:
-------------------------------------
| tape label: first file number = 1 |
-------------------------------------
| directory: A B C D E F G |
-------------------------------------
| data of file A |
-------------------------------------
| data of file B |
-------------------------------------
| data of file C |
-------------------------------------
Reel #2:
-------------------------------------
| tape label: first file number = 4 |
-------------------------------------
| directory: A B C D E F G |
-------------------------------------
| data of file D |
-------------------------------------
| data of file E |
-------------------------------------
Reel #3:
-------------------------------------
| tape label: first file number = 6 |
-------------------------------------
| directory: A B C D E F G |
-------------------------------------
| data of file F |
-------------------------------------
| data of file G |
-------------------------------------
Q: I know that many commands like :EDITOR, :SYSDUMP, :COBOL, etc.
actually run program files in PUB.SYS called EDITOR.PUB.SYS,
SYSDUMP.PUB.SYS, COBOL.PUB.SYS, etc. What about :SEGMENTER? Obviously
there can't be a program SEGMENTER.PUB.SYS (file name to long) -- I've
heard somebody say that the :SEGMENTER program file is actually
SEGDVR.PUB.SYS, but when I do a :LISTF on it I see it has only 27
records! Could that be?
Also, :SEGMENTER has a -PREP command in it that seems just like the
MPE :PREP command -- does :SEGMENTER call MPE to execute the -PREP
command? I tried using the COMMAND intrinsic to do a :PREP from my
program, but I got a CI error 12, "COMMAND NOT PROGRAMATICALLY
EXECUTABLE". Perhaps it's MPE's :PREP command that calls SEGMENTER's
-PREP (I can't believe that HP would have written the same code in two
places); if so, can :ALLOCATEing SEGDVR.PUB.SYS make my :PREPs faster?
A: Consider, if you will: A humble HP3000 user trying to fathom the
intricacies of the MPE SEGMENTER. He believes that typing an MPE
command gets him into SEGDVR.PUB.SYS, but, in reality, that innocent
"-" prompt is just the first signpost of
THE TWILIGHT ZONE!
OK, let's try to straighten all this out.
First of all, typing :SEGMENTER is essentially equivalent to just
saying
:RUN SEGDVR.PUB.SYS
(just like :EDITOR = :RUN EDITOR.PUB.SYS). You're right so far.
Next, as you may have guessed, SEGDVR, with its 27 records, just
doesn't have what it takes to execute all the thirty-odd commands that
the SEGMENTER subsystem supports, from -PREP to -ADDSL to -COPY. Much
as I dislike judging a program by how big it is, 2500 or so words of
code aren't enough to do all that.
All that SEGDVR actually does is PARSE the commands typed at the
"-" prompt. Once it determines what command was typed and what the
parameters were, it passes it to a system-internal procedure called
SEGMENTER (not the command, but a procedure in the system SL). MPE's
:PREP, incidentally, also calls the SEGMENTER procedure, with exactly
the same parameters as :SEGMENTER's -PREP.
So, now things are simple. :SEGMENTER is just a parser -- the code
to actually do all the dirty work is in the system SL, right? Wrong.
The SEGMENTER procedure itself is only about 200-odd instructions.
All that it does is that it takes its parameters -- things like the
command number, the procedure/segment being manipulated, the -PREP
capabilities, maxdata, etc. -- and sticks them into a buffer which it
then sends (using the SENDMAIL intrinsic) to a program called
SEGPROC.PUB.SYS. The first command you execute from :SEGMENTER causes
this process to be created as a son of SEGDVR.PUB.SYS (that's why the
first command you type in :SEGMENTER is always slower than the
subsequent ones); similarly, when you do a :PREP in MPE,
SEGPROC.PUB.SYS is created as a son process of the CI. In either case,
the SEGMENTER procedure is used to communicate with the SEGPROC
process using the SENDMAIL and RECEIVEMAIL intrinsics.
Thus, if you want to make sure that your :PREPs go as fast as
possible, you should :ALLOCATE SEGPROC.PUB.SYS. To speed up your
:SEGMENTERs, you should :ALLOCATE both SEGPROC.PUB.SYS and
SEGDVR.PUB.SYS.
Finally, as you pointed out, although you can easily do a
programmatic :RUN (using the CREATEPROCESS intrinsic) and a
programmatic compile (by CREATEPROCESSing the appropriate compiler in
PUB.SYS), it's harder to do a programmatic :PREP. My suggestion would
be to put the PREP command into a file and then CREATEPROCESS
SEGDVR.PUB.SYS with its $STDIN redirected to that file. Alternatively,
you might call the SEGMENTER procedure directly (or even create
SEGPROC.PUB.SYS as a son process and communicate with it using
SENDMAIL and RECEIVEMAIL), but neither of these approaches is
documented by HP, and might change without notice.
:SEGMENTER :PREP
| /
v /
SEGDVR.PUB.SYS /
(just a parser) /
\ /
\ /
--------------\ /---------/
\ /
v
SEGMENTER procedure (system SL)
|
|
(via SENDMAIL/RECEIVEMAIL)
|
|
v
SEGPROC.PUB.SYS
(the workhorse)
Q: Today I came across a strange phenomenon upon which you may be able
to shed some light.
I have two files on disc which contain identical data (I checked by
running FCOPY on them with the COMPARE option), so I would assume that
they would occupy the same number of sectors on disc. But I was WRONG!
As you can see from the following :LISTF, the "vital statistics" of
the files are the same:
FILENAME ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
TN3WS 140B FA 3000 6000 20 3311 8 8
TN3WS86 140B FA 3000 6000 20 1672 4 8
but the numbers of sectors are different!
What's up?
A: The point that you raise is a good one. Clearly the system needs
only 4 extents for your 3000 records; why is it that one of your files
has all 8 extents allocated?
There are two possible reasons for this:
* In the :BUILD command (and in the equivalent options of the FOPEN
intrinsic), you can specify both the MAXIMUM NUMBER OF EXTENTS A
FILE SHOULD HAVE (in your case, it's 8 for both files) and HOW
MANY OF THOSE EXTENTS ARE TO BE ALLOCATED INITIALLY. You might
say, for instance
:BUILD X;DISC=100,10,3
and have a file that looks like:
FILENAME ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
X 128W FB 0 100 1 33 3 10
Although all the file HAS to have is 1 extent (to fit the file
label), your :BUILD command requested that 3 extents be initially
allocated. Similarly, saying
:BUILD TN3WS;REC=-140,20,F,ASCII;DISC=6000,8,8
will build the file
FILENAME ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
TN3WS 140B FA 0 6000 20 3311 8 8
which has all 8 extents allocated.
Why would you want to initially allocate more extents than
absolutely necessary? Well, if you only allocate one extent to
start with, then as you're writing to the file, new extents will
be allocated as necessary. If you run out of disc space when an
FWRITE operation requires a new extent to be allocated, the
FWRITE will fail -- thus, it would be possible for your program
to fail halfway through its execution, having written 3000 of
6000 records but having no room on disc to write the remaining
3000. If you allocate all the extents initially, then any "OUT OF
DISC SPACE" condition will be reported when you :BUILD the file;
once the file is built, you're guaranteed that all 6000 of your
FWRITEs will succeed.
* The other reason why a file would have more extents than its EOF
seems to indicate is that although new extents are allocated when
needed, they are NOT deallocated when they're no longer
necessary.
Say that you fill up your 6000-record file, causing all 8 extents
to be allocated; then, you open it with OUT access, wiping out
all the file's records. The no-longer-needed 7 extents are not
deallocated by this operation. Thus, your file might have once
been full; then someone opened it with OUT access (throwing away
all the records in the file, but not throwing away the extents)
and wrote 3000 new records to it. The file's number of extents
thus reflects the maximum number that was ever needed for this
file, even though that many may no longer be needed now.
Well, those are the two most likely explanations for you. If you want
to do something about the wasted space occupied by the empty extents,
you might rename the file, :FCOPY it, and then purge the old copy. The
new file will be built by :FCOPY to have only one extent to start
with; then, new extents will be allocated as needed, but never more
than necessary. Alternatively, if you use VESOFT's MPEX/3000, you
could just use the %ALTFILE command:
%ALTFILE TN3WS, EXTENTS=8
which will automatically rebuild the file, freeing all the extents
that are allocated but not used.
Q: My program DBOPENs a database that I know may be redirected with a
:FILE equation. How can I find out the TRUE filename of the database?
In other words, if my program opens database MYDB, and the user types
:FILE MYDB=SUPERB.TEST.PROD
how can my program figure out that it really is SUPERB.TEST.PROD that
it's accessing?
A: There are several ways of finding the fully qualified name of the
root file of the actual database being accessed by your program.
One solution -- the non-privileged one -- rests on the principle
that if there's a :FILE equation for your database root file, you can
get at it using LISTEQ2 (on MPE IV), LISTEQ5 (on MPE V), or the MPE
:LISTEQ command (which, I believe, exists starting with about T-MIT).
You can, for instance, use the COMMAND intrinsic to do a :LISTEQ into
a disc file, and then scan the disc file for a :FILE command that
applies to your database.
A second solution, which requires privileged mode, is based on the
fact that a root file is an MPE file that can be FOPENed just like any
other MPE file. To open a database file, all you need to do is:
* Be in privileged mode when you FOPEN it (AND when you do any
other file system operation, such as an FGETINFO or FCLOSE
against it).
* Specify the exact filecode of the file to be opened (for root
files, this is -400) in the FOPEN call.
Thus -- in SPL terminology -- you can say something like this:
GETPRIVMODE;
FNUM:=FOPEN (DBFILENAME, 1 << old >>,, ,,, ,,, ,,, -400);
IF FNUM<>0 THEN
BEGIN << successful FOPEN >>
FGETINFO (FNUM, REAL'FILENAME);
FCLOSE (FNUM, 0, 0);
END;
GETUSERMODE;
As you see, you FOPEN the root file using whatever filename your
program normally uses and then use FGETINFO to return you the REAL
filename of the file (fully-qualified, of course).
The advantage of this approach is that it's easier than doing
:LISTEQ into a file (or, on pre-T-MIT systems, running LISTEQ2 or
LISTEQ5 with output redirected) and then parsing the file. The
disadvantage is, of course, that it's privileged, and also that
THE USER CAN ISSUE A :FILE EQUATION SUCH AS
:FILE MYDB;DEL
THAT WILL CAUSE YOUR PROGRAM TO PURGE THE ROOT FILE!
The FOPEN will open the file MYDB but then fall under the influence of
the user's file equation, which will cause the FCLOSE to purge the
file! DBOPEN won't let the user do that (because it uses a system
internal privileged procedure to see exactly what parameters were
specified on the :FILE equation), but your ordinary unprotected FOPEN
won't.
Finally, there is a third solution, also privileged. Whenever you
DBOPEN a database, the root file is FOPENed on your behalf by the file
system. For any FOPENed file, you can call FGETINFO against its file
number and get its true filename (although for a privileged file, you
have to be in privileged mode to call FGETINFO). The trouble is that
to do this, you have to know the root file's file number, which DBOPEN
does not return to you. DBOPEN does set the first two bytes of the
database name to be the "database id", but that isn't related to the
file number.
What you can do, though, is scan ALL THE FILES THAT YOUR PROCESS
HAS OPENED until you come across one whose filecode is -400 (the code
of a root file). Then -- provided that your process has only opened
one database -- that will be the file number of the opened database's
root file.
Your program will then look something like this:
FOR FNUM:=3 << min file# >> UNTIL 255 << max file# >> DO
BEGIN
GETPRIVMODE; << FNUM may be privileged >>
FGETINFO (FNUM, FILENAME,,,,,,, FILECODE);
IF = << FNUM is a valid file number >> AND
FILECODE=-400 << A root file >> THEN
BEGIN
GETUSERMODE;
<< we have the right filename >>
...
END;
GETUSERMODE;
END;
As you see, you go through all valid file numbers, from 3 (the lowest
file number you can have) to 255 (the highest); if the file number is
invalid, FGETINFO will set the condition code to < -- if it's valid,
it will set the condition code to =, and then you can check the file
code to see if it's an IMAGE root file.
All things considered, the first approach (LISTEQ into a disc file)
is the most general (though quite cumbersome) -- it doesn't endanger
your database at all, and it works regardless of how many databases
you have open. The third approach (FGETINFOing all the file numbers)
works only if you have one database open. I would recommend against
the second approach (FOPENing the root file yourself), since then --
by accident or by design -- the root file could be injured or deleted.
Q: Help! I have several files that I just can't get at. I try to
copy them, :PURGE them, FOPEN them, and what-not; I always get
prompted for the lockword. If I go into LISTDIR5 and do a >LISTF
;PASS, LISTDIR5 shows them as having lockwords of "/"; however, if I
specify a "/" in response to the lockword prompt, MPE gives me a
LOCKWORD VIOLATION.
I thought that lockwords always had to be alphanumeric. How did
my file get this "/" lockword? How can I remove it?
A: You've fallen victim to a somewhat unusual MPE file system bug
that I've encountered a couple of times in the past. If you call the
FRENAME intrinsic and pass it a filename of, say, "NEWNAME/" instead
of just "NEWNAME", the FRENAME intrinsic creates the new file not
with an empty lockword, but with a lockword of "/". This will only
happen if you use the FRENAME intrinsic -- MPE's :RENAME command will
not allow you to specify a filename with a "/" but no lockword.
To open the file, you'll have to FOPEN it with a filename of
"NEWNAME/" -- again, a slash but no lockword. As before, MPE won't
let you specify such a filename, so you can't just say ":PURGE
NEWNAME/"; however, you can say
:FILE SALVAGE=NEWNAME/
and then say
:PURGE *SALVAGE
or
:RENAME *SALVAGE,NEWNAME
So, a strange bug but a simple workaround. Oddly enough, though,
there IS a very good use for a filename with a slash but no lockword.
Say you want to FOPEN a file but you DON'T want to prompt for a
lockword (perhaps your terminal is in block mode and a prompt would
only mess things up). If the file has no lockword, you'd much rather
have the system immediately return an FSERR 92 (lockword violation)
and not output anything to the terminal. To do this, just call FOPEN
with a filename of "FILE/.GROUP.ACCT". If the file has no lockword
(or a lockword of "/" caused by the bug mentioned above), the system
will open it; if the file does have a lockword, the system will
return an error but won't prompt the user for the lockword. You can
then, for instance, prompt for the lockword yourself using V/3000, or
perhaps look up the lockword in some data file or something.
Thus, to summarize:
File has File has File has bad ("/")
no lockword lockword lockword
"XXX"
FOPEN A.B.C OK Prompts Prompts, but any answer
is rejected
FOPEN A/XXX.B.C Error 92 OK Error 92
FOPEN A/.B.C OK Error 92 OK
Opening a file with a "/" but no lockword is useful for both
opening files with "/" lockwords (caused by the MPE error) and for
avoiding lockword prompts for files that have normal lockwords.
Q: What's the difference between $CONTROL PRIVILEGED, OPTION
PRIVILEGED, and OPTION UNCALLABLE? I always thought that $CONTROL
PRIVILEGED would make all the procedures in my program run in PM, but
it seems that it doesn't. What's going on?
A: First of all: what are OPTION PRIVILEGED and OPTION UNCALLABLE?
These are both SPL keywords that are placed in a procedure header,
and both of them have to do with PM capability; however, there is a
world of difference between them. Consider the following SPL
procedure:
PROCEDURE P;
OPTION UNCALLABLE;
BEGIN
...
END;
This simply means that ANYBODY WHO CALLS P MUST CALL IT FROM WITHIN
PRIVILEGED MODE. P is "UNCALLABLE" from normal user mode.
This can apply to procedures inside a program file but is more
often used in SL procedures. For instance, the system SL contains a
Now the very point of system SL procedures is that they can be called
from user programs -- DBOPEN, FREAD, PRINT, etc. are all system SL
procedures; however, we certainly don't want user programs to call
the SUDDENDEATH SL procedure. Since SUDDENDEATH is declared as
OPTION UNCALLABLE, only code that is running in privileged mode --
whether in the system SL or in a program file -- can call it. Any
other calls would cause the calling program to abort with a PROGRAM
ERROR #17: STT UNCALLABLE.
OPTION PRIVILEGED, however, does almost the opposite. Instead of
indicating that the CALLER of this procedure must be privileged, it
indicates that the procedure itself is to always execute in
privileged mode, REGARDLESS OF WHETHER OR NOT ITS CALLER IS
PRIVILEGED. Thus, saying:
PROCEDURE P;
OPTION PRIVILEGED;
BEGIN
...
END;
means that regardless of whether P's caller is in privileged mode or
not, P's code will always execute in PM. What's more, MPE doesn't
really keep the PRIVILEGED/ non-PRIVILEGED information on a
procedure-by-procedure basis (although the UNCALLABLE/non-UNCALLABLE
information IS kept for each procedure). An entire SEGMENT is either
permanently privileged (all its code executes in privileged mode) or
non-privileged (all its code executes in whatever mode its caller was
in). An OPTION PRIVILEGED procedure "taints" the entire segment it's
in, causing all procedures in the segment to always execute in
privileged mode (in which no bounds checking is done and system
failures are easy to cause).
At this point, we might also mention that what we're talking about
here is PERMANENTLY PRIVILEGED mode. Code that calls GETPRIVMODE and
GETUSERMODE WILL ONLY BE IN PRIVILEGED MODE BETWEEN THE "GETPRIVMODE"
AND THE "GETUSERMODE" CALLS. However, if we use OPTION PRIVILEGED
(or $CONTROL PRIVILEGED) instead of GETPRIVMODE/GETUSERMODE calls (as
is usually done for SL procedures), then entire segments will be
marked as permanently privileged.
$CONTROL PRIVILEGED is exactly like an OPTION PRIVILEGED for the
program's "outer block". Say that you have a program that has no
procedures in it -- just the main body. $CONTROL PRIVILEGED will
indicate that the main body itself will be permanently privileged,
just as a procedure's OPTION PRIVILEGED indicates that the procedure
will be permanently privileged. If your program has several
procedures in several segments, a $CONTROL PRIVILEGED will indicate
that THE SEGMENT THAT CONTAINS THE PROGRAM'S OUTER BLOCK is
permanently privileged; other segments need not be unless they have
OPTION PRIVILEGED procedures in them.
A $CONTROL PRIVILEGED thus does NOT apply to the entire source
file; it only applies to the outer block and, by extension, to the
segment that contains the outer block.
So, the important distinctions to remember are:
* OPTION PRIVILEGED (procedure executes in priv mode) vs. OPTION
UNCALLABLE (procedure can only be called from priv mode);
* permanently privileged (where the entire segment is always
privileged) vs. temporarily privileged (where you're only in
priv mode between the GETPRIVMODE call and the GETUSERMODE
call);
* and $CONTROL PRIVILEGED (which indicates that the segment
containing the outer block is privileged) vs. OPTION PRIVILEGED
(which indicates that the segment containing this procedure is
privileged).
Finally, one important note: the Intrinsics Manual describes two
intrinsics (GETPRIVMODE and SWITCHDB) as "O-P", or "option
privileged" (see p. 2-1 of the FEB 1986 edition). This does NOT
really mean "option privileged"; most intrinsics (including FOPEN,
QUIT, etc.) are declared as OPTION PRIVILEGED because they must
execute in privileged mode. Rather, the Intrinsics Manual's "option
privileged" means two things:
- for SWITCHDB, it really means "OPTION UNCALLABLE"; if you try to
call SWITCHDB from user mode, your program will abort with an
STT UNCALLABLE error;
- for GETPRIVMODE, it really means neither OPTION UNCALLABLE nor
OPTION PRIVILEGED; rather, it means that the calling program
must have been :PREP'ED WITH CAP=PM, regardless of whether or
not it is actually in privileged mode at the time of the call
(which is the distinction that OPTION UNCALLABLE uses);
Keep this in mind and be aware that neither of these definitions are
the true meaning of OPTION PRIVILEGED.
Q: What is the "COLD LOAD ID" number that LISTDIR5 outputs with its
LISTF command? It seems to be the same as the "cold load identity
field" in the :LISTF ,-1 output, but I can't for the life of me figure
out what it means.
A: A file's file label contains a lot of data about the file -- the
file's name, its filecode, the locations of its extents on disc, etc.
As a general rule, pretty much everything that FOPEN might need to
know about a disc file before opening it is kept in its file label.
Say that you FOPEN a file for exclusive access. All subsequent
attempts to FOPEN the file should fail, since you have it open it
exclusively -- but how is the system going to know this? Well,
whenever a file is accessed in any way (by FOPEN, by :RUN, by :STORE,
or by :RESTORE), the appropriate bit in the file label is set
indicating the type of access. Subsequent access attempts will look at
those file label bits to see if the file is being accessed in an
incompatible way. This is what prevents concurrent exclusive FOPENs,
:PURGEs of running programs, :STOREs of files that are being modified,
and so on. There are in fact several such fields in the file label --
fields that are used to indicate the way in which a file is being
accessed:
* The store bit, restore bit, load bit, exclusive bit, read bit,
and write bit (word 28);
* And, the so-called FCB vector (words 32-33) that indicates where
an FOPENed file's control block is located in memory -- this way,
two people who FOPEN the same file can easily share its file
control block.
Now, so far, all this has nothing to do with the coldload id.
However, ask yourself this: What if the system goes down while a file
is being accessed?
The file's access bits and FCB vector are still set to indicate the
file is in use; when you reboot the system, the file will apparently
be in the process of being accessed. Further attempts to FOPEN the
file exclusively will fail, shared FOPENs will try to share the FCB
pointed to by the (now entirely invalid) FCB vector, and so on.
Once upon a time there was a bug like this in IMAGE -- the IMAGE
root file had its Data Base Control Block's data segment number stored
in one of its records (to make it easy for subsequent openers of an
IMAGE databases to find the DBCB and share it); when the system
crashed and was then rebooted, IMAGE still thought that the DBCB was
stored in that data segment, which now didn't exist or contained
entirely different data. Naturally, this caused a good deal of grief.
Therefore, there must be some way for the system to determine
whether the "transitory" data in the file label -- data that is set
when a file is open and is supposed to be reset when the file is
closed -- is really correct or just a carry-over from a previous
"incarnation" of the system. Since by the very definition of a system
failure the system can't close the files normally (as opposed to a
normal =SHUTDOWN, which will normally close the files), there must be
some other solution.
One idea that comes to mind is to, at system startup time, go
through all the file labels in the system and reset their "transitory"
data. Unfortunately, if a system has 30,000 files and can perform
about 30 disc I/O's per second, this would take 1000 seconds or
somewhat over 15 minutes. Not a totally unreasonable amount of time,
but not very quick, either. When a system has just crashed and you're
rebooting it, what you care about most is getting it back up as
quickly as possible, and not taking time going through the labels of
files which, for the most part, may have not been accessed in months.
This is where the cold load id comes in. It's not really a cold
load id, but rather a "restart id"; whenever a system is rebooted
(with a warmstart, coolstart, cold load, update, or reload), this
number is incremented; whenever a file is opened, the current cold
load id is put into the file's file label.
This way, whenever you FOPEN a file, the file system has a very
simple way to see if the transitory data in the file label is valid --
if the current system cold load id is equal to the cold load id in the
file label, then the data is valid; if it is different, this means the
file has not been accessed at all since the last system re-start, and
even if the file label indicates that the file is in use, this is only
a carry-over from a previous system incarnation which crashed while
the file was open.
Thus, what you need to know about a cold load id is:
1) You should never really have to care about it, since everything
the system does with it is behind your back;
2) It is used for making sure that, when the system crashes with a
file open, subsequent incarnations of the system will not think
the file still open because of the access bits that may still be
set in the file label;
3) It should properly be called not a "cold load id" but a "restart
id", since it's reset at every system re-boot, from a warmstart
on up.
Actually, there are circumstances in which you may be able to use
cold load id's yourself. Say that you have a program that keeps some
data in a file while the file is opened -- for instance, you want to
make it easy for people to see who's doing what to the file, so every
time your program opens the file, it writes the opening user's name
into some special place, and every time it closes the file, it deletes
the user's name.
What happens when the system crashes and is then re-booted? Why,
the file contains the names of all those people who were using the
file when the system crashed but are obviously no longer using it now.
You need some way of figuring out whether this data was written to the
file before or after the last system failure.
What you can do is, every time you put some of this "transitory
data" (in this case, the file user's name) into the file, also put in
the current cold load id. Then, when you look at this data, you can
check the current cold load id against the one stored in the file --
if it doesn't match, this means that the transitory data in the file
was written before the last system re-start and is thus obsolete.
The only other thing you need to know is how to get the current
system cold load id. There's no intrinsic that gets you this
information, but there is an (undocumented) non-privileged procedure
that can get it for you:
INTEGER PROCEDURE SYSGLOB (WORDNUM);
VALUE WORDNUM;
INTEGER WORDNUM;
OPTION EXTERNAL;
This procedure lets you get an arbitrary value from the system's
"SYSGLOB" area, and it so happens that word %75 (decimal 61) is the
cold load id. Thus, just saying (in SPL)
INTEGER PROCEDURE SYSGLOB (W); VALUE W; INTEGER W; OPTION EXTERNAL;
...
COLDLOADID:=SYSGLOB(%75);
or (in FORTRAN)
INTEGER SYSGLOB
...
ICOLDLOADID = SYSGLOB (\%75\)
or (in COBOLII)
CALL "SYSGLOB" USING \%75\ GIVING COLD-LOAD-ID.
will get you the cold-load id.
The following is a re-run of a Q&A that appeared in the March 1987
issue of Interact. I'd like to print it again because it describes a
rather mysterious-seeming situation that is all too easy to get into;
in fact, recently I've had some people call me and ask me about this
very problem.
Q: On several occasions I've had very difficult problems caused by
"garbage" characters (escapes, nulls, control characters, etc.).
Sometimes they're in my source files; sometimes they're in my data
files; in any case, they're very hard to see at first glance and even
DISPLAY FUNCTIONS doesn't show some of them (e.g. nulls).
I'd like to have a job stream that goes through a few of my most
important files and checks to see if there are any garbage characters
inside them. How can I -- preferably without writing a special
program -- have a job stream check a file for garbage characters?
A: This is a classic "MPE PROGRAMMING" question -- how to do a
seemingly complicated task without having to write a special program
to do it. As always, MPE programming problems require a good deal of
ingenuity and, to be frank, a rather twisted way of looking at
a problem.
As I'm sure you're aware, :FCOPY has an option called ;CHAR that
outputs the contents of the ;FROM= file REPLACING ALL GARBAGE
CHARACTERS BY DOTS. For instance, if your ;FROM= file line is "AB"
followed by a null (ascii 0) followed by "CD", then an
:FCOPY FROM=MYFILE;TO;CHAR
will output the line as
RECORD 0 (%0, #0)
00000: AB.CD
The null character was replaced by a ".".
Now, FCOPY may have this option, but of what good is it to us? If
FCOPY had, for instance, set some JCW to 1 if at least one garbage
character was replaced by a "." and to 0 if none were, then we'd be
home free -- all we'd have to do is do the :FCOPY ;CHAR, check the
JCW, and that's that.
Unfortunately, FCOPY has nothing like this; it just takes the
;FROM= file and copies it (with whatever transformations are
appropriate) to the ;TO= file.
Here is where we have to be tricky. First of all, we have to
remember that FCOPY has another (less well-known) parameter called
;NORECNUM. This merely says that (when you specify ;CHAR, ;OCTAL, or
;HEX) the output should NOT contain the "RECORD 0" or "00000:"
headers. Thus, an
:FCOPY FROM=MYFILE;TO;CHAR;NORECNUM
will output the line
AB.CD
without any record numbering information.
So, this is what we do. First we say:
:BUILD MYFILENG;REC=<<must be the same as MYFILE>>
:FCOPY FROM=MYFILE;TO=MYFILENG;CHAR;NORECNUM
"MYFILENG" is now EXACTLY the same as MYFILE except that all garbage
characters have been replaced with "."s. Now, we say
:FCOPY FROM=MYFILE;TO=MYFILENG;COMPARE
This will now compare MYFILE and MYFILENG to see if there are ANY
DIFFERENCES between the two files. Obviously, these differences will
ONLY exist if there were some garbage characters in MYFILE (since all
non-garbage characters are copied exactly). Fortunately, if the
:FCOPY ;COMPARE finds at least one difference, it sets JCW to be
equal to FATAL (in batch). Therefore, we can say:
:JOB FINDGARB,ROGER.DEV;OUTCLASS=,1
...
:BUILD MYFILENG;REC=<<must be the same as MYFILE>>
:FCOPY FROM=MYFILE;TO=MYFILENG;CHAR;NORECNUM
:SETJCW JCW=0
:FCOPY FROM=MYFILE;TO=MYFILENG;COMPARE
:IF JCW<>0 THEN
: TELL ROGER.DEV !!! MYFILE has garbage characters !!!
:ENDIF
:PURGE MYFILENG
:EOJ
In fact, by looking at the job stream $STDLIST, you can even find
out in what record and at what word in the record the first
discrepancy (i.e. the first garbage character) occurred -- FCOPY
prints this information for you. Note, however, that you shouldn't
try to find all the garbage characters by doing, say, an ":FCOPY
;COMPARE=100". If you specify a maximum number of compare errors
(100 in this case), FCOPY won't actually set JCW unless at least that
many errors were found.
Finally, note that the file you're examining in this way should
contain only ASCII data -- if it's a data file that contains binary
data (e.g. word 5 is viewed as an integer), that data may appear as
"garbage" to FCOPY, since it views the entire input record as ASCII.
Also note that "garbage" means to FCOPY any character with an ASCII
value of 0 to 31 or 127 to 255 -- this includes all the ASCII
characters with the parity bit set, and may also include various
"native language character set" characters.
Q: With the advent of Spectrum we are using PASCAL to write systems
level code that formerly would have been written in SPL.
In SPL there was a close correlation between program statements and
machine language instructions, but in PASCAL, it is often difficult
to tell how small and/or efficient the resulting code will be. The
PASCAL/3000 manual is of limited help regarding these concerns.
For example, there are four ways of appending string B to string A:
(1) A := A + B;
(2) STRAPPEND (A, B);
(3) STRMOVE (STRLEN(B), B, 1, A, STRLEN(A) + 1);
(4) STRWRITE (A, STRLEN(A) + 1, T, B);
All four of these statements should do the same thing (assuming that
there's no error, i.e. STRLEN(B) > 0 and STRLEN(A) + STRLEN(B) <=
STRMAX(A)). Which of them will generate the "best" object code;
which will generate the "worst"? Will the answer to this question be
any different on MPE/XL?
A: You raise a very interesting question. Indeed, in many high-level
languages a single simple-looking operation can actually do a vast
amount of work and take a long time to do it. In fact, you needn't
look as far as a third-generation programming language -- do you know
how much stuff a simple PCAL (a one-word instruction) has to do, and
how long it might take? If it needs to swap in the code segment from
disc, this innocent-looking instruction might easily take you 30
milliseconds or more!
There is one sure way to find out which of the above four
operations is the fastest -- try them and see. The program I hacked
up looks like this:
$STANDARD_LEVEL 'HP3000'$
PROGRAM TIMINGS (INPUT, OUTPUT);
VAR I, T, TIME: INTEGER;
A, B: STRING[256];
FUNCTION PROCTIME: INTEGER; INTRINSIC;
BEGIN
TIME:=PROCTIME;
FOR I:=1 TO 10000 DO
BEGIN
SETSTRLEN (A, 30);
SETSTRLEN (B, 30);
END;
TIME:=PROCTIME-TIME;
WRITELN ('CONTROL':20, TIME);
TIME:=PROCTIME;
FOR I:=1 TO 10000 DO
BEGIN
SETSTRLEN (A, 30);
SETSTRLEN (B, 30);
A:=A+B;
END;
TIME:=PROCTIME-TIME;
WRITELN ('A+B':20, TIME);
TIME:=PROCTIME;
FOR I:=1 TO 10000 DO
BEGIN
SETSTRLEN (A, 30);
SETSTRLEN (B, 30);
STRAPPEND (A, B);
END;
TIME:=PROCTIME-TIME;
WRITELN ('STRAPPEND':20, TIME);
TIME:=PROCTIME;
FOR I:=1 TO 10000 DO
BEGIN
SETSTRLEN (A, 30);
SETSTRLEN (B, 30);
STRMOVE (STRLEN(B), B, 1, A, STRLEN(A)+1);
END;
TIME:=PROCTIME-TIME;
WRITELN ('STRMOVE':20, TIME);
TIME:=PROCTIME;
FOR I:=1 TO 10000 DO
BEGIN
SETSTRLEN (A, 30);
SETSTRLEN (B, 30);
STRWRITE (A, STRLEN(A)+1, T, B);
END;
TIME:=PROCTIME-TIME;
WRITELN ('STRWRITE':20, TIME);
END.
As you see, we use the PROCTIME intrinsic to determine the number of
CPU seconds consumed by the program so far; a PROCTIME before and
after each operation will tell us exactly how much CPU time the
operation took.
Note also that I had to make an arbitrary assumption -- I assume
that both strings are 30 characters long. Quite possibly one of the
methods might have a longer "start-up" time but be faster for each
character; in that case, copying longer strings might make that
method more efficient, while copying shorter strings might make it
less efficient.
Finally, note that none of the loops includes JUST the operation
we need to test. For one, we have to reset both strings to their
initial values; for another, the FOR loop itself takes a non-trivial
amount of time (to increment the variable, compare, and branch).
This is why the first loop is a "control" loop intended to figure out
how long everything EXCEPT the concatenation actually takes.
So, what's the result? Well, before I ran the program, I decided
to do a little self-test -- I guessed what I thought the relative
rankings would be. You might want to try to do this, too.
Intuitively, my general idea is "the more general-purpose a
function, the longer it'll take". This is because a function that
does only one thing can make all sorts of efficiency-improving
assumptions that the general function can't make.
It seems that the function most uniquely adapted to this job is
STRAPPEND. Whatever the fastest way of appending two strings is,
there's no earthly reason why STRAPPEND couldn't use it. Thus, it
stood to reason that the STRAPPEND (method (2)) would be the best.
My guess for second place was the A := A + B; third place went to
the STRMOVE. Don't ask me why I didn't guess the other way around --
perhaps STRMOVE's five parameters scared me.
I was absolutely sure that the STRWRITE would be the slowest of
them all. This is probably because I've been biased by FORTRAN's
formatter (the dog to end all dogs). General-purpose formatting
facilities are very useful (although PASCAL's is quite ugly), but
they're almost never very fast.
So that was my "educated" guess. What were the results on my
MICRO/XE?
CONTROL 257
A+B 1791
STRAPPEND 1073
STRMOVE 2042
STRWRITE 15733
The numbers are in milliseconds, and indicate the time taken to do
each 10,000-iteration loop. Thus, each STRWRITE took about 1.6
milliseconds.
Curiously enough, my intuition was rather confirmed by the test.
(I guessed before I saw the numbers -- scout's honor!) STRWRITE was
even slower than I though; the other three were pretty even, but
STRAPPEND (the least general of them) was the champ.
At this point, I asked a Spectrum-possessing friend of mine to run
the same test on his machine. Since the one thing he's forbidden to
do on his machine is test Spectrum performance, I will protect him by
leaving him anonymous. Let's just call him "Deep Code".
The results with the Spectrum Native Mode PASCAL/XL compiler were
somewhat different:
CONTROL 1
A+B 11.8
STRAPPEND 20.1
STRMOVE 6.5
STRWRITE 58.8
(To prevent comparisons between Spectrum and MPE/V, all the times are
given relative to the control time -- no absolute timings should be
inferred.)
As you see, the cumbersome-seeming STRMOVE was actually faster
than either the A+B or STRAPPEND. STRWRITE was still way behind.
Finally, I decided to compile the program with range checking off
($RANGE OFF$) on MPE/V. The results were now:
CONTROL 256 (with $RANGE ON%: 257)
A+B 1737 (with $RANGE ON$: 1791)
STRAPPEND 1070 (with $RANGE ON$: 1073)
STRMOVE 875 (with $RANGE ON$: 2042)
STRWRITE 15697 (with $RANGE ON$: 15733)
As you can see, the results are mostly unchanged EXCEPT for STRMOVE,
which is now the fastest of the lot, more than twice as fast as
before!
So there's an answer -- STRAPPEND is fastest on MPE/V with $RANGE
ON$, STRMOVE is fastest with $RANGE OFF$ or on MPE/XL. This is an
answer, but I don't think that it's THE answer.
In this day and age, the most valuable commodity in a DP
department is not computer time -- it's programmer time. If all you
care about is efficiency, you might as well stick with assembly code.
Consider also that the efficiency of most of your code is largely
irrelevant; the overwhelming majority (90% or more) of your program's
time is spent in less than 10% of its code.
My general philosophy, then, is this:
* DESIGN FOR EFFICIENCY.
* CODE FOR SIMPLICITY.
* OPTIMIZE LATER.
When you're making fundamental decisions about your algorithm
(sequential access vs. direct access, B-trees vs. hashing, etc.),
efficiency should play a major role in your thinking -- once you've
committed to one fundamental approach that later proves too slow,
it's often too difficult to change to another approach.
However, when I write my code, I almost invariably choose the
simplest, most straightforward approach -- for reliability,
readability, and maintainability. It so happens that it's about 50%
faster to shift right by 1 bit than to divide by 2. However, if I
want to average two numbers, I'd much rather say:
AVERAGE:=(MAXIMUM+MINIMUM)/2;
than
AVERAGE:=(MAXIMUM+MINIMUM)&ASR(1);
Looking at the first statement, I instantly see what's going on -- it
documents itself. To understand the second statement, I really have
to think about it. Similarly, even if it were faster to say
something like
AVERAGE:=DIVIDE(ADD(MAXIMUM,MINIMUM),2);
(where DIVIDE and ADD are presumably super-efficient built-in
functions), I'd still rather say
AVERAGE:=(MAXIMUM+MINIMUM)/2;
The (MAXIMUM+MINIMUM)/2 is just a lot easier to read and understand.
What's more, chances are that any little optimization I might do
to this statement would be of virtually no consequence. As I said
before, the overwhelming majority of a program's time is spent in a
very few places. You can spend weeks writing all your code in the
most "efficient" manner (and months fixing the bugs caused by all the
extra complexity), when you could have just spent a few hours
optimizing those few statements that take the most time.
What's more, I've found that I ALMOST NEVER know what parts of the
program will actually be the most frequently used ones. I just write
everything in the simplest, cleanest way and then use HP's wonderful
APS/3000 program to find out where the "hot spots" are. APS/3000 can
tell you EXACTLY where your program is spending most of its time --
armed with this data, you can often change a half dozen lines and get
a two-fold performance improvement.
Therefore, my recommendation is to use whatever mechanism looks to
you to be the easiest and most understandable. I'd recommend that
you say
A := A + B;
because that seems to me to be the clearest solution. Similarly,
even though STRWRITE is so horribly slow, I'd still recommend that
you use it in cases where it best represents what you're doing (for
instance, if you want to use the ":fieldlength" syntax or you want to
concatenate numbers as well as strings).
Then, after you've written the program, run APS/3000. If you find
that this statement is in a particular tight loop and is taking a
large portion of the program's run time, then you can optimize it to
your heart's content. However, don't spend too much effort chasing
an instruction here and there. Your time is a lot more valuable than
the computer's. First, a few words in response to some of the Letters
to the Editor in the February 1988 issue.
Mr. Gerstenhaber of Computation & Measurement Systems in Tel-Aviv
commented on avoiding EOF's on PASCAL READLNs in case the input starts
with a ":". He quite correctly points out that if you issue a file
equation :FILE INPUT=$STDINX (the "X" somehow dropped off the :FILE
equation in the printed copy of his letter), PASCAL will read from
$STDINX and will not get an error on ":" input.
My original answer recommended that you put in a "RESET (INPUT,
'$STDINX ');" statement into your PASCAL program -- this solution has
the advantage of making the program completely stand-alone, so it
doesn't require a :FILE equation to run properly. Mr. Gerstenhaber's
solution, on the other hand, has the advantage of not requiring any
source code changes (although the program won't run quite right
without a :FILE equation). Both solutions are very reasonable
alternatives.
Mr. van Herk of Mentor Graphics in the Netherlands inquired about
saving scheduled/waiting jobs across a coldload and about "letting a
session log itself off after a certain idle period". These issues were
apparently the subjects of previous Q&A questions (which were answered
by Mr. N. A. Hills).
The first question was originally asked in the June 1987 Q&A -- a
user did a coldload once a week (as recommended by HP), and found that
his scheduled jobs were deleted. He couldn't just re-submit them,
since they were originally submitted by users -- the system manager
doesn't know the original job stream filenames, and in any case the
original files might have already been modified or purged.
As Mr. van Herk quite correctly points out, the contributed library
JSPOOK program addresses this very problem. I'm not sure exactly how
much it preserves -- whether it works for WAITing jobs only, or
SCHEDuled ones as well; I suspect that there are newer and older
versions of JSPOOK in various places that do slightly different
things.
However, JSPOOK is definitely the place to start -- look for it on
the contributed library; it might very well do exactly what you want.
"Letting a session log itself off after a certain idle period" is a
different story. I guess that what the user wanted was to abort
sessions whose user has walked away from his terminal (thus causing a
potential security threat) or is just signed on and not doing anything
(which may, for instance, use precious ports on your port selector).
Q: I've always wondered how IMAGE calculates the maximum number of
extents for a dataset. Most of my datasets end up having 32 extents,
but some smaller datasets have fewer. Why is this? Is it some sort of
MPE limitation, or is it IMAGE's own choice.
A: My old IMAGE Manual (March 1983 version) has little to say on this
topic: "Each data file is physically constructed from one to 32
extents ..., as needed to meet the capacity requirements of the file,
subject to the constraints of the MPE file system" (p. 2-10).
To get the answer, I had to decompile the IMAGE code in the system
SL. As best I could tell from the machine instructions, the algorithm
was:
#extents = flimit / 64 + 1
In other words, if the file limit ends up being 920 records (the file
limit having been calculated from the capacity, the record size, and
the blocking factor), the number of extents will be
920/64 + 1 = 14 + 1 = 15 extents
(note that the division rounds down).
The principle here seems to be to avoid really small extents. The
best reason for this is disc caching (although I think the above
algorithm might have been chosen before disc caching was implemented)
-- the most that MPE can cache is one extent; if extents are very
small, you'll lose much of the benefit of caching.
Usually, these numbers of extents should work quite well for you.
If for some reason you want to change them, though, you can't just
issue a :FILE equation at DBUTIL >>CREATE time (since >>CREATE ignores
file equations). One thing you might do is use MPEX's
%ALTFILE datasetfilename;EXTENTS=newnumextents
(%ALTFILE ;EXTENTS= lets you change the maximum number of extents for
any disc file). However, as I said, you'd rarely want to do this.
Q: I'm trying to call a PASCAL procedure from a FORTRAN program and I
get a loader error that says "INCOMPATIBLE FUNCTION FOR" and then the
name of the PASCAL procedure. I know that PASCAL is a strict
type-checking language, but my FORTRAN calling sequence seems to
exactly match the PASCAL procedure's formal parameter list! (The
PASCAL procedure returns an integer and takes several simple integer
by-reference parameters, which is exactly what I pass to it.) What's
going on here?
A: The compilers, the segmenter, and the loader support a little-known
mechanism known as the CHECK LEVELS. This allows MPE (when :PREPping
together separately compiled code or when calling an SL procedure) to
make sure that the caller and the callee both have the same idea of
how the procedure parameters should be passed.
The procedure that is called (the "callee") may have (in its USL,
RL, or SL) information describing its parameter types; the procedure
that calls it (the "caller") may have information describing the
parameter types that it expects. If the two don't match, an error
message is printed.
By default, SPL generates all its procedures (and all its procedure
calls) with OPTION CHECK 0, which indicates that NO CHECKING IS TO BE
DONE. Thus, if you try to call an average system SL procedure (almost
certainly written in SPL), no parameter checking will be done -- if
you mis-specify it in an OPTION EXTERNAL declaration, you're in deep
trouble.
Now, FORTRAN and PASCAL by default generate all their procedures
(and all procedure calls) with OPTION CHECK 3, which indicates that
the NUMBER OF PARAMETERS, THE RESULT TYPE, and EACH PARAMETER TYPE are
to be checked. If FORTRAN generates an OPTION CHECK 3 procedure call,
but the called procedure is OPTION CHECK 0, no checking will be done;
if SPL generates an OPTION CHECK 0 procedure call, but the called
procedure is OPTION CHECK 3, no checking will be done. However, if
both the procedure call and the procedure itself are OPTION CHECK 3,
full checking will be done to make sure that the caller's and callee's
procedure declarations are identical.
So, what's the big deal? After all, you say that the FORTRAN call's
parameters are perfectly compatible with the PASCAL procedure's header
-- even though there's an OPTION CHECK 3 test, it should be passed
with flying colors.
Well, if you look in chapter 9 of your System Tables Manual
(what??? you say you don't have a System Tables manual???), you'll see
the following paragraph:
"PASCAL: Pascal sets the high order bit in the parameter type
descriptor when it is generating hashed values. The remaining 15
bits are based on a hash of the types of the parameter. Only the
Pascal compiler can compute the value, and the SEGMENTER must
match the whole 16 bit value."
Since PASCAL has so many different types (RECORDs, enumerated types,
subranges, etc.), the PASCAL compiler generates a special "parameter
type entry" into the procedure's OPTION CHECK 3 descriptor, an entry
that is INCOMPATIBLE WITH THE ENTRIES GENERATED BY ANY OTHER COMPILER,
even if the parameter types referred to by PASCAL and the other
compiler are perfectly compatible.
In other words, you can't call (with OPTION CHECK 3) from a
non-PASCAL program any OPTION CHECK 3 PASCAL procedure. What you must
do is either:
* make sure that the PASCAL procedure is NOT compiled with OPTION
CHECK 3, or
* make sure that the calling program does not generate any
procedure calls with OPTION CHECK 3.
PASCAL has two compiler keywords:
* $CHECK_ACTUAL_PARM$, which indicates the checking level for
PROCEDURES YOU CALL, and
* $CHECK_FORMAL_PARM$, which indicates the checking level for
PROCEDURES YOU ARE DECLARING.
FORTRAN, on the other hand, has only one relevant compiler keyword:
* $CONTROL CHECK=, which indicates the checking level for
PROCEDURES YOU ARE DECLARING.
What this means is that FORTRAN will ALWAYS GENERATE ITS PROCEDURE
CALLS WITH OPTION CHECK 3, since there's no keyword to turn this off.
Therefore, you must change your PASCAL procedure to have a
$CHECK_FORMAL_PARM 0$.
What if you don't have the source code to the PASCAL procedure? In
this case, you're in trouble, since the procedure has check level 3
and FORTRAN will always generate external procedure calls with check
level 3. To avoid this, you have to write an SPL "gateway" procedure
which calls the PASCAL procedure and is called by the FORTRAN
procedure. If the SPL gateway procedure declares the PASCAL procedure
to be OPTION CHECK 0, EXTERNAL, this will work.
For the curious: Yes, there are check levels 1 and 2. OPTION CHECK
1 indicates checking of only the PROCEDURE RESULT TYPE; OPTION CHECK 2
indicates checking of the PROCEDURE RESULT TYPE and the NUMBER OF
PARAMETERS. OPTION CHECK 3, as I mentioned before, checks the
PROCEDURE RESULT TYPE, the NUMBER OF PARAMETERS, and the TYPES OF ALL
THE PROCEDURE PARAMETERS.
Q: I notice that MPE lets me allocate extra data segments that belong
to a process (call GETDSEG with id = 0) or a shared within a session
(GETDSEG with id <> 0). I want to have a global data segment that's
shared among many sessions. I'd like to be able to have one program
that loads this segment up from a file or from a database (say, at
the beginning of the day), and all the other ones will then read data
from the segment without having to go to disc. How can I do this?
A: Unfortunately, you can't share data segments among sessions
without doing some serious privileged mode work. However, are you
sure you really want to have an extra data segment?
Whenever people talk about accessing files, they think "disc I/O".
After all, files are stored on disc, right? Well, almost right. When
you read a disc file (in default, buffered mode), the file system
doesn't actually go out to disc for every FREAD; rather, it reads one
disc BLOCK at a time (however many RECORDS it may contain) into an
in-memory buffer. Whenever the record to be read is already in the
buffer, it's taken from the buffer rather than from disc. This is
then just a memory-to-memory move, rather like a DMOVIN.
Now, if you want to keep your data in a data segment, you must
have no more than 32K words of data (actually, slightly less).
Curiously enough, this also happens to be close to the maximum size
of a file system buffer. If you say
:BUILD X;REC=128,255
then each block in X will be built with 255 records of 128 words
each. When you first read the file, the file system will read all
32640 words of it into memory; any subsequent reads of the file (as
long as they fit within those 128 records) will require NO DISC I/O
AT ALL, since they'll be satisfied entirely from the in-memory
buffer.
Thus, what you need to do is:
* Build the file with a sufficiently high block size (record size
WILL FIT IN ONE BLOCK.
* Always open the file SHR (shared) GMULTI -- the GMULTI means
that everybody will share THE SAME BUFFER (rather than having
one 32K data segment for each file accessor!).
Then, all of your reads will be (almost) as fast as data segment
accesses, since that's exactly what they'll be! The only difference
is that you'll be letting the file system take care of all the data
segment management behind your back.
In fact, using files in this case will give you a lot of other
advantages (besides inter-session sharing) over data segments:
* You can use your language's built-in file access mechanisms
instead of having to call DMOVIN, DMOVOUT, GETDSEG, FREEDSEG,
etc.
* If you want to look at your file to make sure that it's correct,
you can use FCOPY, EDITOR, etc.
* If people will be updating the data segment, you can use FLOCK
and FUNLOCK to coordinate the write access to it. (Make sure,
however, that nothing you do relies on the current record
pointer -- see below!)
* If the system goes down, the file will still remain around.
In general, files are just a lot easier to deal with than extra
data segments, and as long as you block them right (and stay within
32K, which is the limit for a data segment anyway), they can be as
fast or almost as fast!
The only thing you need to beware of is that when you open a file
GMULTI, not only the file buffers are shared among all the file
accessors, but so are the current record pointers! In other words, if
two people have a file opened GMULTI and both are reading it
sequentially, one will get about half of the file's records and the
other will get the rest! As soon as one reader reads record #0, the
current record pointer (which both readers share) will be incremented
to 1, and the other reader will read record #1 and never see record
#0.
Therefore, whenever you're reading (or writing) a GMULTI file,
DON'T DO SEQUENTIAL READS (unless you do some sort of locking, for
reads as well as writes). Do all your I/O using FREADDIR and
FWRITEDIR (or their equivalents in your language).
Q: Sometimes, when I do a :LISTF of a file, I get a display like
this:
FILENAME CODE ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
URLDAT USL 128W FB 141 959 1 60 2 32
The file has 529 records, but uses only 300 sectors! Is this some
super compression algorithm HP is using? Can I make all my files this
small?
A: Sorry, no. This file uses only 300 sectors because it has less
than 300 records worth of actual data. Although there is a record
#528 (which is why the EOF is 529), some of the records between
record #0 and record #528 are missing!
How can this happen? Well, try it yourself. Build a file with a
command such as
:BUILD X;DISC=1023,8
-- a file with 1023 records and up to 8 extents. The first of these
extents will be allocated when you build the file (except on MPE/XL);
the others will be unallocated until you write to them.
Now, write a small program that does an FWRITEDIR of a record into
record #1022. Then, when you :LISTF the file, you'll see something
like:
FILENAME CODE ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
X 128W FB 1023 1023 1 256 2 8
Only the first extent and the last extent (the one with record #1022)
will actually be allocated -- the others will remain empty until you
actually write to them.
Once you think about it, it's all pretty straightforward -- you
write a record to an extent and that extent gets allocated; if you
don't write a record to an extent, it won't get allocated. However,
when you first see this, it can be puzzling indeed!
In fact, this situation is quite rare, since most people either
write to files sequentially or entirely allocate the whole file to
start with (e.g. IMAGE databases, KSAM key files, and most data
files). The SEGMENTER is one of the few subsystems that regularly
creates "holey" files; it's quite common to find USL files like this.
Q: There's a program that I run that HAS to have a temporary file of
its own to do its work -- it just couldn't possibly work otherwise.
However, when I hit [BREAK] and do a :LISTFTEMP, MPE says there are
no temporary files; :LISTF doesn't show me any new permanent files
either. Where is the program keeping all its temporary data? It
doesn't have DS capability, so I know it's not in an extra data
segment. I'm not quite clear on the distinction between permanent
and temporary files, anyway. Are they just kept in two different
places?
A: If you want a file to be available to sessions other than its
creating one (either at the same time or later), you must make it a
PERMANENT file. The system will then keep its name and a pointer to
its data in the PERMANENT FILE DIRECTORY, a large chunk of disc space
reserved for this very purpose. :LISTF, of course, scans through the
directory, as does FOPEN when you ask it to access an existing
permanent file.
If you create a file within your own session and know that you'll
never need it outside your session, you can make it a TEMPORARY file.
Its name and data pointer will be kept in the TEMPORARY FILE
DIRECTORY, an extra data segment (actually part of the JDT) kept by
the system on your session's behalf. (Naturally, there is one
Temporary File Directory for each session in the system, but only one
Permanent File Directory for the entire computer [ignoring for a
moment private volumes].)
Temporary files have two major advantages: first, they go away
when your session dies (and their disc space is released) -- this can
be convenient, since otherwise you'd have to remember to purge them
before logging off (naturally, you'd always forget).
More importantly, since each session has its own Temporary File
Directory, you need not be afraid of interfering with another
session's temporary files. If your program needs to build a work
file (which you want to call WORKFILE) and two people are running it
in the same group, then you have the possibility of both processes
trying to create WORKFILE at the same time. If you keep WORKFILE as
a temporary file, each session will have its own WORKFILE file in its
own Temporary File Directory.
Temporary files also have two major disadvantages. One, of
course, is the very fact that they are temporary -- if the system
crashes or the job aborts, the files will be lost; there'll be no way
of seeing what was in them. If one of your job temporary files
contains important information, this may be a serious concern -- if
your job aborts, the file will be lost, and you won't be able to see
what was in the file (which might give you some important clues as to
why the job aborted).
The other problem is that when the system crashes, the space used
by the temporary files is lost. Since the Temporary File Directories
are all kept in extra data segments, the information in them
(especially the pointers to the file data) will be lost as well, and
thus the system will not be able to re-use the space occupied by the
temporary files until you do a RECOVER LOST DISC SPACE.
However, as your question points out, there are more things than
just permanent and temporary files. "Permanent" and "temporary"
indicate what DIRECTORY the file name and the pointer to the file
data are kept in -- what if they don't need to be kept anywhere?
What if a file will only be needed while a program is running, and as
soon as the program closes it, it can safely be discarded? In this
case, you wouldn't need to keep pointers to it in any permanent or
even semi-permanent place -- only in the file tables that MPE keeps
in your own stack.
When you call FOPEN (or when it's called on your behalf by your
language's compiler library), you can tell FOPEN which directory to
look for the file in. You can tell it to look in the PERMANENT FILE
DIRECTORY -- this means that the file must exist as a permanent file
at FOPEN time; you can tell it to look in the TEMPORARY FILE
DIRECTORY -- this means that the file must exist as a temporary file
at FOPEN time. Finally, if you don't expect the file to exist at all
but rather want to create a new file, you can indicate that in the
FOPEN call, too.
When you call FOPEN to open this "new file", the file will NOT
actually be cataloged in either directory -- this will only happen
when you FCLOSE the file (you can tell FCLOSE whether to save the
file as permanent or temporary). As long as the file is FOPENed but
not yet FCLOSEd, the file exists (space is allocated for it on disc),
but it's not pointed to by ANY directory. If the process FCLOSEs the
file without asking FCLOSE to save it, the file will go away;
similarly, if the process dies without FCLOSEing the file, it'll go
away as well. The file is thus a "process temporary file", in that
it is accessible only by the process that created it (unless the
process later saves the file) -- in the manuals, it's usually called
a "new file", meaning not just a recently created file but rather a
file that is not kept track of in either the Permanent or the
Temporary File Directory.
This is what is almost certainly happening in your mystery
program. It needs to store data somewhere, but sees no reason to
save it for access by other sessions or even by other processes in
the same session. It FOPENs the file as NEW, and then, when it's
done with the data, either FCLOSEs the file (without saving it) or
just terminates. Neither :LISTF (which looks in the Permanent File
Directory) nor :LISTFTEMP (which looks in the Temporary File
Directory) will show this file, because the file is utterly unknown
to anybody other than the creating process.
We can summarize all this with the following table:
PERMANENT FILE TEMPORARY FILE NEW FILE
--------- ---- --------- ---- --- ----
Available to Anyone, any time Only creating Only creating
session process
Shown by :LISTF :LISTFTEMP Nothing
If process File remains File remains File destroyed,
aborts space recovered
If job aborts File remains File destroyed, File destroyed,
space recovered space recovered
If system File remains, File destroyed, File destroyed,
crashes space not wasted space lost space lost
Q: I recently did a :SHOWME in one of my batch jobs, and I got some
rather unusual output:
USER: #S329,TEST.PROD,PUB (NOT IN BREAK)
MPE VERSION: HP32033G.B3.00. (BASE G.B3.00).
...
$STDIN LDEV: 4 $STDLIST LDEV: 5
I don't have ldevs 4 and 5! I expected it to say "$STDIN LDEV: 10"
(since 10 is my :STREAMS device) and "$STDLIST LDEV: 6" (since 6 is
the only device in my LP class), but it gave me those rather bizarre
values instead. What's going on here?
A: Actually, if you did have ldevs 4 and 5 configured, you would NOT
have gotten them in your :SHOWME output. In fact, a :SHOWME in a
spooled (i.e. :STREAMed to a spooled printer) batch job is GUARANTEED
to give you nonexistent $STDIN LDEV and $STDLIST LDEV numbers!
Why is this? Well, in the depths of the "System Operations and
Resource Management" manual (p. 7-92 of my January 1985 edition), you
can find an interesting statement:
"When a spool file is opened, MPE creates a 'virtual device' of
the required type by filling in an unused logical device entry
with the appropriate values."
What does this mean?
Well, say that you open a spool file on device class LP. At first
glance, you might think that the device number associated with this
file would be 6, the device of the line printer. Actually, this
clearly can't be so -- when you write a record to the spool file,
device 6 doesn't actually get written to; rather, the spool file on
disc is written to and is then (when the file is closed) copied to
device 6.
OK, you think, that's right -- the file is on disc; therefore, its
device number must be 1, 2, or 3 (the device numbers of my disc
drives), depending on which disc drive it happened to be built.
Nope. For some reason, the operating system requires that the spool
file be more than just a simple disc file. Perhaps this is because
some MPE I/O (e.g. the CI's prompt for the command) goes through the
internal ATTACHIO procedure rather than through the file system; the
ATTACHIO call requires a logical device number.
In any event, whenever you open a spool file, the system assigns a
"virtual logical device number" to it, and most access to the file
then happens through this device; naturally, this virtual logical
device number must not be the same as any real device number. Since
all batch jobs (except those that are submitted from non-spooled
devices, such as tapes and card readers, or those that are sent to
non-spooled devices, such as hot printers) have spool files both for
their $STDINs and $STDLISTs, all batch :SHOWMEs and WHOs will return
these virtual (i.e. invalid) logical device numbers.
If you're curious about getting the REAL $STDIN and $STDLIST
devices, call the new JOBINFO intrinsic asking for items 9 (input
device) and 10 (output device). The :SHOWME command should really do
this, but it was written long before JOBINFO was implemented.
Q: My program opens a file with Input/Output access, does some reads
from it, some writes to it, and then closes it. When I run the
program it works well, but when one of my users runs it, nothing
happens. The program doesn't get a file open error, but the file
doesn't get updated, either. I thought it might be a security
violation, but the file open's succeeding, so that can't be it.
A: Well, maybe it can. One would think that if you don't have write
access to a file, an open for Input/Output would fail with an FSERR
93 (SECURITY VIOLATION); however, this is not so. If you have read
access but NOT write access and you try to open a file for
Input/Output (or Update), the FOPEN will SUCCEED; however, the file
will be opened for READ ACCESS ONLY.
Now, if you try to write to the file, the FWRITE will fail (with
an FSERR 40, OPERATION INCONSISTENT WITH ACCESS TYPE). If you
checked for a file system error after the FOPEN but you DON'T checked
for a file system error after the FWRITE, everything will have looked
OK; but, since the FWRITE failed, the file won't have been updated.
What can you do? Well, you could (and should) check for an error
after the FWRITE call; if, however, you want to detect the error
condition at FOPEN time, you have to do something like this:
FNUM:=FOPEN (FILENAME, 1 << old >>, 4 << in/out >>);
IF <> THEN
FILE'ERROR
ELSE
BEGIN
FGETINFO (FNUM, , AOPTIONS);
IF AOPTIONS.(12:4)<>4 THEN
<< you asked for IN/OUT access, but got something less >>
...
END;
The FGETINFO call will get you the REAL aoptions that the file was
opened with -- then you can check to see if the access granted was
really IN/OUT.
Moral of the story (#1): ALWAYS check for error conditions.
Moral of the story (#2): Sometimes a successful result isn't
really successful.
Q: I have a program that runs just fine when I run it normally (with
$STDIN not redirected). However, when I try to redirect its $STDIN
to a disc file, it aborts on MPE with a tombstone that says "ERROR
NUMBER: 42". I looked it up, and the manual says it's "OPERATION
INCONSISTENT WITH DEVICE TYPE (FSERR 42)". What does that mean? The
program doesn't use any special devices (tapes, printers, etc.).
A: Well, actually the program DOES use a special device -- your
terminal. When you run your program with its $STDIN redirected to a
disc file, all of its file system operations on $STDIN become
operations on a disc file (rather than on a terminal). For normal
FREADs, that's OK -- you can read from a disc file just as well as
from a terminal. But what if the program does an FCONTROL mode 13 on
$STDIN to turn off echo? Or an FCONTROL mode 14 to turn off break?
These operations work when passed a terminal file; however, when you
pass them a disc file (even if its your program's ;STDIN=), they'll
fail with the very error condition you indicated.
Oddly enough, if your program is doing, say, an FCONTROL mode 13
to turn off echo, then the file system error is actually no problem.
The FCONTROL 13 is only useful when $STDIN is a terminal; if it's a
disc file, the FCONTROL can't be done, but it isn't needed, either.
It may be that your program is seeing this error and aborting
although it would be perfectly OK for it to go on.
If that's what you're doing -- an FCONTROL mode 13, or perhaps one
of the other FCONTROLs that's only needed when you're really reading
from the terminal -- then you may want to check for an error
condition, and if it's FSERROR 42, keep going as if nothing had
happened. Or -- dare I suggest it -- maybe not even check for the
FCONTROL error at all?
Moral of the story: Sometimes an error really isn't an error.
Q: I'm doing a "SQUEEZE" of a disc file (i.e. releasing allocated but
unused disc space beyond the EOF) with an MPEX %ALTFILE ;SQUEEZE; I
understand that this just opens the file and closes it with
disposition 8. This works for all my fixed record length files and
some of my variable record length files, but some variable record
length files it leaves completely untouched. Their FLIMITs and disc
space usages remain exactly the same as before. What's happening?
A: Well, I was stumped until I looked at my trusty MPE source code.
After I saw the actual code that did the "squeeze", everything fell
into place.
Once upon a time (before MPE IV came out), FCLOSE mode 8 only
worked for fixed record length files. This is because its job is to
release all the unused blocks beyond the end of file -- for this, it
has to know what the last used block is. In a fixed record length
file, the last used block # (counting from block #0) is equal to
CEILING (eof/blockingfactor) - 1
because every block has exactly blockingfactor records.
In a variable record length file, however, there can be any number
of records in a block. The EOF (the number of records in the file)
won't tell you what the last block # is; the only way to find the
last block (in MPE III) was to read the file sequentially until the
EOF was reached -- way too slow for FCLOSE 8 to use.
MPE IV allowed you to append to variable record length files
(perhaps because message files, which are always variable record
length, needed to be appended to). To do an append, the file system
also needs to know the last block #; for this, the file system
started keeping the last used block # in every variable record length
file's file label. This incidentally made FCLOSE mode 8s of variable
record length files possible. Thus, in MPE IV and later systems, an
FCLOSE mode 8 of a variable record length file simply goes to the
"end of file block number" and throws away all the blocks that go
after it.
So why do some FCLOSE 8s succeed and others do nothing? Well,
what if the file system sees an end of file block number of 0? It
could mean a file with exactly 1 block (block #0) -- OR, it could
mean a file that was first created on a pre-MPE IV system, when the
end of file block# field was ALWAYS set to 0! FCLOSE 8 can't just
throw away all blocks after block #0, since there MAY be (on a
pre-MPE IV file) a lot of data in those blocks.
Therefore, you have a bit of a paradox. Any variable record
length file that has at least two data blocks will get properly
squeezed; however, any file that's very small -- has only one data
block -- won't be modified. In this isolated case, the smaller files
may actually use more space (when squeezed) than the larger ones!
Q: Is there an easy way to determine what language a program was
written in? Maybe some word in word 0 of the program file?
A: Nothing that direct, I'm afraid. Code is code -- a program file
is just a set of an assembly instructions; there's no need for the
operating system to inquire into where they came from, so MPE doesn't
keep this information around.
However, if there isn't a direct way, there may well be an
indirect one. In fact, there are two -- neither is certain, but they
work for most programs.
The first method is based on the fact that all compilers -- except
SPL -- generate calls to so-called "compiler library" procedures.
For instance, when you do a WRITELN in a PASCAL program, the compiled
code won't actually contain all the instructions that do the I/O, nor
even a direct call to the FWRITE or PRINT intrinsic; rather, the code
will have a call to the P_WRITELN procedure, a system SL procedure
that actually does the PASCAL-file-to-MPE-file mapping, the I/O, the
error check, etc. All languages (except SPL) rely on such compiler
library procedures, typically to do I/O, but also to do bounds
checking, implement certain complicated language constructs, etc.
(Even SPL has a few "compiler library" procedures of its own that it
calls -- anybody know what they are?)
The point here is that each language has its own compiler library
routines -- PASCAL's typically start with "P'" (e.g. P'WRITELN),
COBOL's with "C'" (e.g. C'DISPLAY), BASIC's with "B'" (e.g.
B'PRINTSTR). You'd think that FORTRAN's would start with "F'", but
they don't -- the most common ones are FMTINIT', SIO' (and in general
xxxIO'), and BLANKFILL'.
Armed with this knowledge, you can just run the program with a
;LMAP parameter. This will show you all the external references,
which, in addition to those intrinsics that you explicitly call,
should include a whole bunch of compiler library procedures. From
the names of these procedures, you can figure out the source language
of the program (if there are no such procedures, it's probably in
SPL).
Note that it is possible, for instance, for a FORTRAN program NOT
to call any compiler library procedure (if it does no formatting and
nothing else that requires any compiler library help) -- however,
it's quite unlikely.
The other way of figuring out a program's source language is by
using the fact that all languages (except for SPL) do some sort of
automatic I/O error checking. If they encounter an unexpected error
condition, they will abort with tombstones -- each one with its own
kind. The trick is to force an I/O error; for a program that's doing
character mode I/O, the easiest way is by typing a :EOD on $STDIN.
* Most FORTRAN programs that do an ACCEPT or READ (5,xxx) will
abort with "END OF FILE DETECTED ON UNIT # 05" and a
PRINTFILEINFO tombstone for file "FTN05".
* Most COBOL programs will print an error message such as "READ
ERROR on ACCEPT (COBERR 551)" and then do a QUIT. The "COB" in
the "COBERR" should clue you in.
* Most PASCAL programs will abort with an error message like "****
ATTEMPT TO READ PAST EOF (PASCERR 694)".
* Most BASIC programs will abort with an "**ERROR 92: I/O ERROR ON
INPUT FILE" and then a BASIC tombstone, which is similar to a
PRINTFILEINFO but different. The second line of the tombstone
will probably be "FNAME: BASIN".
This is a less reliable trick since it won't work for programs
that do no direct terminal input (e.g. ones that do no terminal input
at all or do it through V/3000); also, some smart programs may trap
input errors and handle them themselves, rather than just letting the
compiler library abort.
Finally, note that these techniques will probably work for HP
Business Basic, RPG, and maybe even C programs -- all of them will
probably have their own compiler libraries, and all of them (except,
perhaps, C) will have their own way of aborting on an input I/O
error.
Q: I have about 30 job streams that need to run over the weekend. I
want them all to run in sequence, not because each one depends on the
previous ones, but because each is so CPU-intensive that running two
at once would bring the system to its knees (and completely drown out
any other users that might be foolish enough to try to use the
computer at the time).
At first, I tried to set the job limit to 1. This proved
impractical in my environment, since other users may want to submit
their own jobs while the 30 sequential jobs are running. (The other
users' jobs can't use ;HIPRI to bypass the job limit since the users
don't have SM or OP.)
The next thing I tried was the old trick of having each job stream
submit the next job stream at the end of its execution -- for
instance, JOB1 might look like:
!JOB JOB1,USER.PROD;OUTCLASS=,1
...
!STREAM JOB2
!EOJ
JOB1 streams JOB2, JOB2 streams JOB3, etc. Unfortunately, if I do
this, then any job that aborts in the middle would prevent any of the
other jobs from running, which is unacceptable. I thought of putting
!CONTINUEs in front of every line in each job stream, but I WANT a job
stream error to flush the remainder of THAT job stream (but not the
others). Putting !CONTINUEs and then !IFs designed to prevent (in case
of error) the execution of everything EXCEPT for the final !STREAM was
also unreasonable -- my job streams are hundreds of lines long, and
would thus require hundreds of !CONTINUEs and nested !IFs.
Another idea I had was to have each job stream SCHEDULE (not just
stream) the next job at the BEGINNING (rather than the END) of the
job. Thus, JOB1 might look like:
!JOB JOB1,USER.PROD;OUTCLASS=,1
!STREAM JOB2;IN=,3
...
!EOJ
The first line in JOB1 schedules JOB2 to run in 3 hours; if JOB1
aborts, JOB2 would already have been scheduled.
This one almost worked; unfortunately, the run times of the jobs
vary greatly. If JOB1 runs more than the interval time (in this case,
3 hours), then JOB1 and JOB2 would have to run simultaneously, and the
system would grind to a halt. If, however, to counteract this I set
the interval time even higher, then the jobs might just take too long
to run -- even at 3 hours per job, the 30 jobs would take 90 hours
(almost 4 days!). I don't want to have any unused time between job
executions.
(Actually, I only mention this for completeness' sake -- I have an
old Series III and am running MPE V/R, which does not support job
scheduling.)
Finally, there's one other alternative I can think of. I can write
a program that wakes up every several minutes, does a :SHOWJOB into a
disc file, sees if one of my jobs is running, and if none are,
:STREAMs the next one (it would keep track of what the next one should
be). Then I would stream this "traffic-cop" program and it would
submit my 30 jobs.
Unfortunately, I don't relish the task of writing this program,
having it analyze the :SHOWJOB output, execute MPE commands, open
files, etc. I'd like to make do with the minimum possible programming.
Is there an "MPE Programming" solution to this problem that doesn't
require writing a custom program that I'd then have to rely on and
maintain?
A: Wow! You sure are tough to please! When I read your opening
paragraph, I thought of all four of the solutions that you proposed,
but it seems that none of them is good enough for you. Fortunately, I
did manage to come up with another solution, though it took some hard
thinking.
Let's look at what you're trying to achieve. You want JOB2 to start
up as soon as JOB1 finishes (no sooner and no later), whether JOB1
finished successfully or aborted. What mechanism currently exists in
MPE for doing this sort of thing?
Well, there's no way of doing EXACTLY this; however, there is a
similar feature not for jobs, but for files. If JOB1 has an empty
message file opened and JOB2 is waiting on an FREAD against this file,
then JOB2 will awaken as soon as JOB1 closes the file, whether the
close is done normally or as the result of a job abort. Thus, if JOB1
said
!JOB JOB1,...
!OPEN MSGFILE FOR WRITE ACCESS
!STREAM JOB2
....
!EOJ
and JOB2 said
!JOB JOB2,...
!READ MSGFILE
...
!EOJ
then JOB2 would try to read MSGFILE (presumably an empty message
file), see that it's empty, and wait until a record is written to it
OR until all of MSGFILE's writers close the file. JOB1 has MSGFILE
opened for write access as long as it's running; when JOB1 terminates,
MSGFILE will automatically be closed, and JOB2 will automatically wake
up.
Sounds good? It's very much what we want, except for two things --
MPE has no way of opening a file or of reading a file. The "!READ
MSGFILE" can actually be done by saying
!FCOPY FROM=MSGFILE;TO=$NULL
Unfortunately, there's really no documented way for your job to open a
message file and keep it open. A program run by the job can open the
message file, but as soon as the program terminates, the file will be
closed; what you want is to have the Command Interpreter process
itself to open the file. But, since the CI has no !OPEN command, this
is impossible, right?
Of course, this (opening a file in the CI and keeping it opened) IS
possible. It just can't be done DIRECTLY, but must instead be done as
a SIDE EFFECT of something else.
The CI often opens files -- the :PURGE command opens the file to be
purged, :RENAME opens the file to be renamed, :LISTF and :SHOWCATALOG
open their list files. However, after opening the file, the CI (being
a well-written program) closes it. If we want to subvert a CI command
so that it will open a file but not close it, we have to somehow
inhibit the close operation.
One way that comes to mind is to somehow make the CI's FCLOSE fail
with some sort of file system error. If the FCLOSE fails, the file
remains opened. Our plan would be to issue some sort of :FILE equation
that would prevent the FCLOSE from succeeding.
There are very few ways in which the FCLOSE of an already-existing
file can fail. One little-known way is if you try to FCLOSE a
permanent file with disposition 2, which means "save the file as
temporary". If you try to do this, you'll get file system error 110,
ATTEMPT TO SAVE PERMANENT FILE AS TEMPORARY (FSERR 110)
If the FCLOSE fails, the file is not closed, and therefore remains
open. Thus, we can say:
:BUILD MSGFILE;MSG
:FILE MSGFILE,OLD;TEMP
:PURGE *MSGFILE
Sure enough, we get the error
ATTEMPT TO SAVE PERMANENT FILE AS TEMPORARY (FSERR 110)
UNABLE TO PURGE FILE *MSGFILE. (CIERR 385)
Now, we do a :LISTF of the file, and we see that... the file has no
asterisk ("*") after its name! Although the FCLOSE failed, the file is
NOT STILL OPENED. The :PURGE command was smart enough to see that,
since the FCLOSE failed, it should do a special FCLOSE that guarantees
that the file will get closed no matter what.
So, what now? Did I lead you all the way through this mess just to
let you down in the end? Hold off on those nasty letters! Try the
following:
:BUILD MSGFILE;MSG
:FILE MSGFILE,OLD;TEMP
:SHOWCATALOG *MSGFILE
Now, do a :LISTF MSGFILE,2 -- the file is still open! The :SHOWCATALOG
command (almost alone of all the MPE commands) does NOT force the
close of its list file; if the FCLOSE fails (as a result of the wicked
:FILE equation we set up), the list file will remain open until the
job or session that did it logs off!
Therefore, here's the solution:
!JOB JOB1,...
!BUILD MSGFILE;MSG
!FILE MSGFILE,OLD;TEMP
!SHOWCATALOG *MSGFILE
!STREAM JOB2
...
!EOJ
!JOB JOB2,...
!FCOPY FROM=MSGFILE;TO=$NULL
!FILE MSGFILE,OLD;TEMP
!SHOWCATALOG *MSGFILE
!STREAM JOB3
...
!EOJ
!JOB JOB3,...
!FCOPY FROM=MSGFILE;TO=$NULL
!FILE MSGFILE,OLD;TEMP
!SHOWCATALOG *MSGFILE
!STREAM JOB4
...
!EOJ
Oh, yes, don't forget to put a few !COMMENTs in those job streams
that explain what you're doing -- I wouldn't want to be the poor
innocent programmer who has to figure out what those !SHOWCATALOGs are
for!
In any event, this is just what you asked for -- an "MPE
programming" solution that meets all your criteria AND doesn't require
a specially-written program that does PAUSEs, SHOWJOBs, etc. The only
problem -- and it's potentially a very serious one -- is that you're
relying on an MPE BUG (:SHOWCATALOG's not closing of its list file
when the normal FCLOSE fails) which in theory could be fixed at any
time.
However, if you're using MPE V/R, you needn't worry about too many
operating system improvements. And, if you can ever ditch your Series
III and get a REAL MACHINE, you might not have such horrible CPU
resource problems.
In any case, this is yet another example of what amazing (and
somewhat disquieting) things that you can do with a bit of ingenuity.
I'll bet you never thought that the :SHOWCATALOG command could do
something like this!
Q: I have a temporary file that I'd like to /TEXT and /KEEP using
EDITOR. I wish that EDITOR had a /TEXT ,TEMP command and a /KEEP ,TEMP
command for doing this, but it doesn't. I tried saying
:FILE MYFILE;TEMP
and then saying
/TEXT *MYFILE
and
/KEEP *MYFILE
-- the /TEXT worked but the /KEEP didn't. I then tried saying
:FILE MYFILE,OLDTEMP
which also let the /TEXT work, but still didn't do the /KEEP right.
Why doesn't this work? What can I do?
A: A :FILE equation's job is to alter the way a file is opened or
closed. Therefore, to understand a :FILE equation's effects on a
particular program, you have to understand how the program opens and
closes the file to begin with.
First of all, what does EDITOR do when you say /TEXT MYFILE? It
FOPENs the file not as an OLD file (which would mean setting bits
(14:2) of the foptions parameter to 1) or as an OLDTEMP file
(foptions.(14:2) := 2), but with foptions.(14:2) set to 3. This
special FOPEN mode tries to look for the file first as a temporary
file, and then if the temporary file doesn't exist, as a permanent
file. When you say
/TEXT MYFILE
EDITOR will first try to text in the temporary MYFILE file, and then
(if there is no temporary MYFILE) as a permanent MYFILE; thus, you
don't have to do anything special to /TEXT in a temporary file. You
didn't need either a :FILE MYFILE,OLDTEMP or a :FILE MYFILE;TEMP.
Now, what happens when EDITOR does a /KEEP MYFILE? It actually does
four things:
1. It tries to FOPEN MYFILE with foptions.(14:2)=3 (look first for
a temporary file, then for a permanent file).
2. If this FOPEN succeeds (i.e. MYFILE already exists), it asks you
"PURGE OLD?", and if you say yes, FCLOSEs it with disposition
DELETE (to purge the old file).
3. It FOPENs MYFILE as a new file (and then writes the data to be
/KEEPed to it).
4. It FCLOSEs MYFILE with disposition SAVE (to save it as a
permanent file).
Assume, then, that there are no file equations in effect. If you don't
have a temporary file (or permanent file) called MYFILE, a /KEEP
MYFILE would:
1. Try to FOPEN the permanent or temporary file MYFILE -- and fail.
2. FOPEN a new file MYFILE and write data to it.
3. FCLOSE MYFILE with SAVE disposition (as a permanent file).
Thus, the /KEEP MYFILE would merely build a permanent file.
What if MYFILE already exists as a temporary file and you do a
/KEEP MYFILE (again without :FILE equations)? EDITOR would:
1. FOPEN the temporary file MYFILE.
2. FCLOSE it with DELETE access (thus purging it).
3. FOPEN a new file MYFILE and write data to it.
4. FCLOSE MYFILE with SAVE disposition.
Thus, you'd still build MYFILE as a permanent file, but you'd also
lose the temporary file MYFILE in the process! (Quite strange,
actually, since EDITOR could easily have saved MYFILE as a permanent
file without touching the temporary file.)
Now, what if there is BOTH a temporary file named MYFILE and a
permanent file named MYFILE? Here's what EDITOR would do:
1. FOPEN the temporary file MYFILE.
2. FCLOSE it with DELETE access (thus purging it).
3. FOPEN a new file MYFILE and write data to it.
4. FCLOSE MYFILE with SAVE disposition -- which would fail because
there's already a permanent file called MYFILE!
In this case, EDITOR purged the WRONG FILE -- purged the temporary
file INSTEAD of the permanent file; you've lost the temporary file and
still haven't done your /KEEP, since the permanent file still exists.
Confused yet? This is what happens WITHOUT a :FILE equation!
Now, let's say that you do a :FILE MYFILE,OLDTEMP when there is no
temporary (or permanent file) named MYFILE. This is what would happen:
1. EDITOR tries to open the file MYFILE (as a temporary file
because of the :FILE equation). The FOPEN fails because the file
doesn't exist, but that's no big deal, since EDITOR expected it
to fail if the file didn't exist.
2. EDITOR now tries to open the file MYFILE as a new file --
however, the :FILE equation overrides this, and the FOPEN
actually tries to open the file as an OLDTEMP file! This FOPEN
fails, and EDITOR prints "*41*FAILURE TO OPEN KEEP FILE (53) --
NONEXISTENT TEMPORARY FILE (FSERR 53)".
If a temporary file called MYFILE existed, you wouldn't have much more
luck, since by the time the second FOPEN took place, the file would
have been deleted by the first FCLOSE.
Finally, let's say that you do a :FILE MYFILE;TEMP when a temporary
file named MYFILE already exists. Then, when you /KEEP *MYFILE, EDITOR
will:
1. Try to FOPEN MYFILE as an old permanent or temporary file --
this FOPEN would succeed quite well, opening the old temporary
file.
2. Ask you whether you want to "PURGE OLD?" -- if you say YES, it
will try to FCLOSE the file with DELETE disposition. HOWEVER,
the ;TEMP on the :FILE equation (which says "close the file as a
temporary file") will override the DELETE -- the file will be
closed, but not deleted!
3. FOPEN MYFILE as a new file and write all the data to it.
4. Try to FCLOSE the file as a permanent file (SAVE disposition).
The :FILE equation's ;TEMP will override this, so the FCLOSE
will try to close the file as a temporary file, which is exactly
what you wanted. However, remember that the purge of the
temporary file that we tried to do in step #2 actually wasn't
done! Thus, the FCLOSE ;TEMP will fail with a "DUPLICATE
TEMPORARY FILE NAME (FSERR 101)". You said "YES" when asked
"PURGE OLD?", but that never happened because of the :FILE
equation.
Those are the problems you've been running into. If this all sounds
complicated, that's because it is. To understand it, you have to be
keenly aware of every FOPEN and every FCLOSE that EDITOR does. Since
these FOPENs and FCLOSEs actually aren't documented anywhere, you
really have to guess at what EDITOR must be doing.
What's the solution? Well, let's see what it is that we WANT each
FOPEN and FCLOSE to do:
WHAT EDITOR DOES WHAT WE WANT IT TO DO
1. Open MYFILE as old Open MYFILE as old
permanent or temporary temporary (so if there's
an old permanent MYFILE
but no old temporary
MYFILE, the old permanent
MYFILE won't be purged)
2. Close MYFILE with DELETE Close MYFILE with DELETE
disposition disposition
3. Open MYFILE as a new file Open MYFILE as a new file
4. Close MYFILE with SAVE Close MYFILE with TEMP
(save as permanent file) (save as temporary file)
disposition disposition
In other words, we want OPEN #1 to be affected by a :FILE ,OLDTEMP
without having the same :FILE equation affect OPEN #3; and, we want
CLOSE #4 to be affected by a :FILE ;TEMP without having the same :FILE
equation affect CLOSE #2.
What single :FILE equation can we use to do this? The simple answer
is: there isn't one. Instead, we must do two things:
/:PURGE MYFILE,TEMP
to delete any old file named MYFILE and then
/:FILE MYFILE;TEMP
/KEEP *MYFILE
to /KEEP MYFILE as a temporary file. Since the temporary file MYFILE
is then already gone, the :FILE ;TEMP will not badly affect FCLOSE #2
(the one that's supposed to do the purge of the old temporary file)
because FCLOSE #2 will not actually be done (since FOPEN #1 would have
failed since the old temporary file did not exist). Instead, the :FILE
equation will influence FCLOSE #4, which will then save the
newly-built /KEEP file as a temporary file rather than as a permanent
file.
Thus, that's your solution:
/TEXT MYFILE << no file equation needed >>
...
/:PURGE MYFILE,TEMP
/:FILE MYFILE;TEMP
/KEEP *MYFILE
The only problem is this: what happens if there is both a permanent
and a temporary file named MYFILE? Will the /KEEP then succeed? The
answer to this question is left for the reader...
Q: I'm trying to decide whether I should do my future development in C
or PASCAL. The one great advantage of PASCAL, I am told, is that it
does much more stringent type checking; I understand that this can be
more of a deficiency than an advantage, but I've heard that PASCAL/XL
lets you optionally waive type checking when needed. I think that type
checking (as long as it can be avoided when necessary) is a very
valuable thing; it's much better to let the compiler catch the bugs
than make the programmer find them all.
However, Draft ANSI Standard C seems to do the converse -- it seems
to bring C's type checking up to a reasonable level (just as PASCAL/XL
brought PASCAL's type checking DOWN to a reasonable level). Is this
true?
A: Like many good answers, the answer to this question is "yes and
no". First, though, let's talk a moment about "classic" (i.e.
pre-Draft ANSI Standard, also known as "Kernighan & Ritchie") C.
K&R C did virtually no type checking at all. For example, say that
the function "func" took two integer parameters. Your program could
easily say
func (10)
and thus pass "func" only one parameter; of course, the results would
be quite unpredictable (and almost certainly undesirable), but the
compiler wouldn't say a thing. Similarly, you could try to pass 3
parameters by saying
func (10, 20, 30)
Again, the compiler will be perfectly happy to do this, even though
it's a pretty obvious bug (one that will probably result in "func"
getting the wrong values and in garbage being left on the stack).
Finally, you might also say:
func (10.0, 20.0)
-- try to pass two floating-point numbers (rather than integers).
Since in most C implementations floating point numbers use twice as
much space as integers, this will again badly confuse both the caller
and the callee, but the compiler will not complain.
Other examples of this total lack of parameter type checking
abound. C requires you to pass all "by-reference" parameters by
specifically prefixing each one with an "&", e.g.
func (&refvara, &refvarb);
If you omit the "&" when it's needed (or specify it when it isn't),
the compiler won't catch the error; needless to say, if you try to
pass a record of type A when the procedure expects a record of type B,
the compiler won't catch this, either.
As you see, this can get pretty unpleasant. God knows, all of us
make mistakes; I, for one, would much rather have the compiler catch
them for me. This may be "protecting the programmer against himself",
but that's a good idea in my book. We programmers need all the
protection we can get, especially against ourselves.
Draft ANSI Standard C -- which more and more C compilers are
implementing -- is much better. It lets you define so-called "function
prototypes", which really are declarations of the parameter types that
each function expects. For instance, if you say
extern int FFF (int, int, float *);
then the compiler will know that FFF is a function that takes three
parameters -- two integers, and a float pointer (i.e. a real number by
reference) -- and returns an integer. Then, if you try to say
float x;
int i, j;
i = FFF (10, 20); or
i = FFF (10, 20, &x, 30); or
i = FFF (10, 20, x); or
i = FFF (10, 20, &j);
then the compiler will catch all four errors (too few parameters, too
many, not-by-reference, and bad type). There are (fortunately) ways to
waive type checking for all calls to a particular function, for a
particular function parameter, or for a particular function parameter
in a particular function call, but in the overwhelming majority of
cases in which type checking is useful, it's available.
So, to answer your question -- is Draft ANSI Standard C type
checking now as good as PASCAL/XL's? In my opinion, I'm afraid it
isn't.
First of all, remember the old "equal sign" problem? One of the
things that has always bedeviled me about C is that C's assignment
symbol is "=" and its comparison symbol is "==". Thus, saying
if (i=5) ...
does NOT check to see if I is equal to 5 -- it assigns 5 to I and
checks to see if the result (the assigned value) is non-zero. Try
finding a bug like this some time -- it certainly isn't easy! People
have told me "oh, you'll get used to it"; I worked with C on and off
for years, and I did NOT get used to it.
Now, many C programmers will tell you that one has no right to
complain about this; after all, it's a fairly arbitrary decision which
symbol is to be used for which function -- C had just as much right to
use "=" and "==" as PASCAL had to use ":=" and "=".
However, the key problem is not C's choice of symbols, but rather
the C compiler's reaction to a programmer error! If you mixed up the
symbols in PASCAL and said something like
IF I:=5 THEN
then the PASCAL compiler would complain, since it doesn't recognize a
":=" inside an IF condition. Even if it did (which would be quite
reasonable), it would still print an error, since the result of "I:=5"
would be an integer and the IF statement would expect a boolean
(logical) value.
Unlike C, PASCAL has a distinction between the integer and boolean
data types, and would thus be able to catch this simple mistake. If C
had the same distinction, it could look at the statement
if (i=5) ...
and realize that if the user really intended to have the IF condition
be the result of "i=5", this would be an error, since the result is an
integer rather than a boolean.
Another thing that PASCAL/XL can do that I suspect no Draft ANSI
Standard C compiler can do is bounds checking. Say that you index
outside the bounds of an array in PASCAL/XL; the code generated by the
compiler will check for this and promptly abort the program. A C
program indexing outside array bounds won't abort unless the index
would be outside the program's data area altogether (which is rarely
the case); instead, it will either return garbage data (if you're
reading), or overwrite random data if you're writing!
In my experience, such bounds-error bugs have by all means been the
most frustrating. Often they don't manifest themselves until you
actually try to use the accidentally overwritten variable, usually
many statements after the actual error took place. These bugs are
notoriously hard to find, and the compiler would do you a great
service if it could help find them for you.
To the best of my knowledge, no Draft ANSI Standard C compiler
supports bounds checking. However, this is not just the fault of the
compiler (I'm sure that there exist some PASCALs that don't do
run-time bounds checking); rather, the language itself makes bounds
checking more or less impossible. Whenever you pass an array to a
procedure, the procedure does not get any information about the array
bounds; whenever you use pointers (which are used far more often in C
than in PASCAL), all possibilities for bounds checking are lost. On
the other hand, PASCAL (and PASCAL/XL) is designed very much with
run-time bounds checking in mind.
Note that, of course, run-time bounds checking exacts a price in
performance (often a very substantial one). However, no-one says that
you should be forced to use it; most PASCAL compilers, including both
PASCAL/V and PASCAL/XL let you turn it off for production code that
you want optimized. What's important is that you should have the
OPTION of using it if you feel it's appropriate.
Finally, a third -- and much less serious -- incompleteness in C's
type checking is that enumerated type constants are viewed as just
another type of integer. For instance, if you say
typedef enum {widget, gadget, thingamajig} item_types;
then you'll still be able to use "widget", "gadget", and "thingamajig"
interchangeably with integers and with other enumerated types. You
could say
i = 10*gadget - widget;
(which is meaningless if "gadget" and "widget" are really supposed to
be symbols rather than integers); more importantly, you can easily
accidentally compare a variable of a different enumerated type against
"gadget", not realizing the difference in types. The PASCAL compiler
would catch this, and tell you that it doesn't make sense to compare
objects of one enumerated type against objects of another, since their
numeric values are (in PASCAL/XL) purely accidental.
So, those are the major ways in which Draft ANSI Standard C's type
checking is less powerful (less likely to catch your bugs) than
PASCAL/XL's:
* No INTEGER vs. BOOLEAN distinction;
* Most importantly, no run-time bounds checking;
* No stringent enumerated type checking.
On the other hand, remember that PASCAL/3000 (and other ANSI Standard
PASCAL's) have much more severe flaws, largely tending towards the
opposite extreme. A few examples are:
* You can't have a procedure that takes a record or array of a
variable type (you have to write one procedure for each different
type -- so much for reusability of code);
* You can't have a procedure that takes arrays of different sizes
(you have to write one procedure for each array size!);
* You can't have a procedure that is actually intended to take
varying numbers of parameters;
* And more...
Where Standard C just makes it easier for you to make errors, Standard
PASCAL makes it almost impossible for you to do certain perfectly
normal and desirable things -- a much graver sin in my book. Imagine
trying to write one matrix multiplication routine for each combination
of array sizes! Or, imagine that you wanted to write, say, a "shell"
procedure for the DBGET intrinsic that would check for error
conditions, possibly call DBEXPLAIN, maybe log something, etc. --
you'd have to write one such procedure for each possible record type
you might want to DBGET!
However, PASCAL/XL seems to solve all (or almost all) of these
problems (as long as you're writing only for MPE/XL and never want to
retrofit your programs to bad old PASCAL/V!). Both Draft ANSI Standard
C and PASCAL/XL seem to be very nice languages, and much good can be
said about both -- however, in the field of error checking, I believe
that PASCAL/XL is somewhat superior.
Q: Whenever I run SPOOK from QEDIT, MPEX, or some such
process-handling environment it seems to "suspend" when I exit it. I'm
not exactly sure what this means; it appears, however that this
"suspension" makes the next :RUN of this program (from the same father
process) faster, plus I have the same file open (at the same current
line number) as I did when I exited!
Other programs, like QEDIT, SUPRTOOL, and MPEX also do this; on the
other hand, QUERY and FCOPY don't. In particular, I'd really like
QUERY to be able to suspend in this way, since I want to be able to
exit QUERY and re-enter it with the same databases opened, the same
records found, etc. Also, I'd like to be able to do the same for my
own programs.
How is this done? Is there some keyword (:RUN QUERY.PUB.SYS;SUSPEND?)
that I can put onto my command to trigger this option? How much extra
resources does all this "suspending" cost me?
A: Normally, when a program decides that it's done (e.g. the user's
typed EXIT), it will call the TERMINATE intrinsic (either directly or
through some language construct, e.g. STOP RUN). The TERMINATE
intrinsic will close all the process's files, clean up any resources
(e.g. locks, data segments, etc.) that the process has acquired,
deallocate the process's stack (which contains all its variables) and
then activate the process's father process.
The son process is gone -- the only way to reactivate it is to
re-:RUN it (or use the CREATE or CREATEPROCESS intrinsic to do the
:RUN programmatically), which will create an entirely different
process, with its own files, variables, etc.
For many programs, this makes perfect sense -- when, for instance,
a compiler is finished compiling, it might very well decide that all
of its internal files, variables, etc. will be of no more use; it
might as well re-activate the father process by TERMINATEing.
However, what about an interactive, command-driven program like
SPOOK? Its job is not to do a specific, isolated task; rather, it is
to execute whatever commands you want it to do. When you type >EXIT in
SPOOK, that might not mean that you don't want to do any other SPOOK
commands; it might just mean that you want to do something else for a
while (e.g. use EDITOR, run QUERY, tc.), and then will want to get
back into SPOOK.
QUERY is an even more aggravated case -- what if you've just
finished a 2-hour >FIND command, and have now discovered that you need
to look at your program to figure out exactly how to, say, format your
>REPORT. You want to get out of QUERY, but only temporarily; you'd
like to go into your favorite editor, look at your program, and then
come back to right where you left off, in QUERY, with the big >FIND
waiting for you.
Fortunately, MPE lets you have more than one process in your
session. You can even have more than one ACTIVE process in the session
(which can get very complicated), but you may certainly have one
active process and several SUSPENDED processes. A son process can --
instead of calling TERMINATE -- merely activate its father and suspend
itself (by calling the ACTIVATE intrinsic with the parameters 0 and
1). Then, the son process and all of its files, variables, data
segments, etc. will stay alive, and the father can (at its discretion)
re-activate the son (using the ACTIVATE intrinsic). Not only will this
preserve the son's internal state, but also be a good deal faster than
re-:RUNing it.
Thus SPOOK has chosen a rather smart solution -- by ACTIVATEing its
father, it gives the father the option of re-ACTIVATEing rather than
re-:RUNing it the next time the user wants to get into SPOOK. Note,
though, that this requires the cooperation OF BOTH THE SON AND ITS
FATHER.
If the son blindly TERMINATEs instead of ACTIVATEing its father,
there's nothing you can do; similarly, if the father doesn't realize
that the son process is not dead (but only sleeping), it may try to
re-:RUN it the next time it's needed, not taking advantage of the
suspended son process that it already has. There's no ;SUSPEND keyword
on the :RUN command (or on the CREATE or CREATEPROCESS intrinsic) with
which a willing father can force an unwilling son process to suspend.
This leaves three questions:
* How can you make your own program suspend whenever possible? In
COBOL, you'd say:
CALL INTRINSIC "FATHER" GIVING DUMMY-VALUE.
IF CC = 0;
THEN CALL INTRINSIC "ACTIVATE" USING 0, 1;
ELSE STOP RUN.
(where CC was declared in the SPECIAL-NAMES section to be equal
to the special name CONDITION-CODE).
The FATHER intrinsic call (to which we pass DUMMY-VALUE, a dummy
S9(4) COMP) will check to see if our father process was the
Command Interpreter or a user process. If it was a user process
(CC = 0), we'll ACTIVATE it, thus suspending ourselves; however,
if it was a CI (CC <> 0), the ACTIVATE call will fail with a
nasty-looking error, so we just do a normal STOP RUN instead.
Of course, after the IF, there should be a GOTO back to the top
of the prompt loop (or something like that); when your process is
reactivated, control will be transferred to the statement
immediately after the IF. It's your responsibility to make sure
that when you're awakened, you'll go on doing what you want to
do.
Remember that it's also the father process's responsibility to
know that a son process has suspended and then re-activate it
instead of re-:RUNing it. If your father process knows that a son
process will suspend, it should save the son process's PIN
(Process Identification Number) that CREATE and CREATEPROCESS
return, and pass it to the ACTIVATE intrinsic (using CALL
"ACTIVATE" USING PIN, 2) when it's time for the son to be
re-activated.
* What about QUERY? Is there anything you can do make it friendlier
in this respect?
For normal QUERY, the answer is, unfortunately, no. QUERY calls
the TERMINATE intrinsic, and that's that. (If you got the idea of
intercepting the TERMINATE intrinsic call and making it call
ACTIVATE instead, no dice -- this will cause QUERY to suspend,
but when QUERY is reactivated, the very next instruction will be
executed, which will cause some rather unpleasant results. QUERY
isn't expecting to get reactivated, so it's not properly prepared
for this.)
Note, however, that if you use our MPEX/3000, our "MPEX hook"
facility lets you make QUERY suspend (and much more!) -- give us
a call at (213) 282-0420 if you can't find "MPEX HOOK" in your
manual.
* Finally, what about performance?
Well, suspended son processes do consume virtual memory and
various table entries (DSTs, PCBs, etc.). However, they do NOT
consume any CPU time or any real memory (since if the memory they
use is needed, they can be safely swapped out).
On the other hand, having your son processes can save you a lot
of time just in TERMINATEs and :RUN alone (each takes some time,
more time than a simple ACTIVATE). Add to this the time you save
re-opening databases and re->FINDing records in QUERY, re-opening
databases and forms files in your own applications, re-/TEXTing
files in EDITOR (which you had to exit since you couldn't just
suspend it), and so on -- you'll find that having your son
processes suspend can be much more computer-efficient as well as
more efficient in using your own time.
Q: I have a very large program that I assemble from several USL files.
After making a few additions, my :SEGMENTER -COPY commands started to
give me "ATTEMPT TO EXCEED MAXIMUM DIRECTORY SPACE" errors; apparently
I overflowed some internal space limitation.
The funny thing is that I've seen some program files that are bigger
than mine (COBOLII.PUB.SYS, for instance, is 1802 records long, while
my program is only about 1600 records long), so I know that it's
possible to have programs that large. What can I do?
A: Every USL file contains two types of information: directory and
non-directory. The non-directory information includes all the actual
machine instructions generated by the compiler; every word of code in
your program takes up at least one word of non-directory information.
The directory information contains only the directory of procedures
that your program contains: one variable-length entry for each
procedure. Although this will almost certainly take up a good deal
less space than the non-directory information, it is also much more
limited -- in fact, you can have at most 32,767 words of directory
information in any single USL file.
This is an ABSOLUTE limit. It's just not possible to fit more than
32,767 words in the USL directory (because the SEGMENTER keeps all
directory pointers in 15-bit fields).
How can you decrease the amount of USL directory space you use? Well,
one approach is to break the USL file up into several files. This can
be done in one of several ways:
* Break the program up into several program files which communicate
via process handling. Naturally, this will only work if there's
some reasonable dividing line in the program -- perhaps if each
program performs several distinct tasks that do not share much
code.
* Move some procedures into an SL file. This will work as long as:
- The procedures do not use any global storage (global
variables, SPL OWN variables, FORTRAN COMMON areas, etc.;
FORTRAN formatting operations are also forbidden in SLs).
- All the procedures you move into the SL call only other
procedures in this SL (or system intrinsics); there's no easy
way of calling from the SL back to your program. In other
words, you'll have to separate your program into SL
procedures and non-SL procedures; the SL procedures can then
only call other SL procedures, but the non-SL procedures can
call either non-SL or SL procedures.
* Move some procedures into an RL file. RL files can, unlike SL
files, use global variables; however, they're limited in some
other ways:
- An RL file can have at most 127 entry points (i.e. procedures
or secondary entry points).
- An RL file can contain at most 16,383 words of code.
- Like SLs, any code you move to the RL must not call code
that's outside of the RL.
An RL can thus only give you a bit of breathing room; it can
decrease your USL directory size by up to 127 entry points' worth
of space; at about 20 words per entry, this can save you about
2,500 words out of the 32,767 maximum.
The best solution will probably be (in most cases) to move procedures
into an SL file. RL files can save you only a little space, and
splitting up a program will often be impossible when all the various
parts of the program share a good deal of code.
Fortunately, there are also a few ways to minimize directory space
usage directly, without splitting your USL file. What exactly does the
directory contain?
* It contains one entry per PROCEDURE, one entry per SECONDARY
ENTRY POINT, and one entry per SEGMENT.
* Each procedure entry (the most common kind) contains 14 1/2 words
of fixed overhead, plus
- the procedure name,
- one extra word if your procedure is OPTION CHECK 1 or OPTION
CHECK 2,
- one word per parameter if your procedure is OPTION CHECK 3,
- one word for each procedure this procedure calls,
- one word for each global variable this procedure references,
- and various other (less important) things.
Some of these things you can't easily control -- you could try to
minimize the number of procedures each procedure calls, but it would
be quite hard. Other things, fortunately, are not so tough:
* The easiest one is the space used for each procedure if they are
compiled with OPTION CHECK 1, 2, or (especially) 3. OPTION CHECK
allows the SEGMENTER to make sure that a procedure is called with
the correct number of parameters and parameter types; in PASCAL
and SPL this essentially means that the SEGMENTER will ensure
that the actual procedure declaration and its OPTION EXTERNAL
declaration are identical.
This is quite valuable if you have many different source files
that you compile into one program; it can prevent a lot of very
strange bugs that might occur because of incompatible procedure
declarations. However, to support this, the compiler must insert
information on all procedure parameter types into the USL, and
that takes up USL directory space.
By default, FORTRAN and PASCAL compile all their procedures with
OPTION CHECK 3, the most thorough but most directory-space-hungry
method. Saying $CONTROL CHECK=0 in FORTRAN or $CHECK_FORMAL_PARM
0$ in PASCAL will switch to the more efficient OPTION CHECK 0.
In SPL, the default is OPTION CHECK 0; however, some people use
OPTION CHECK 3 for the extra error-checking it provides. I used
to do this myself, but found I had to switch back to OPTION CHECK
0 to save space.
* You can try to shorten your procedure names. Long procedure names
are very good because they're much more readable; however,
circumstances may force you to compromise on this point. If
you're running into the USL directory space limitation, you
probably have about 1,000 procedures; if you can shorten each
procedure name by two characters, you can save 1,000 words, which
may mean the difference between success and USL overflow.
* Finally, if you're realy hard up, yuo can combine several
procedures into one. This would be a shame -- one of the
fundamental rules of good structured programming is to make
procedures as small as possible -- but, as I said, you may have
to compromise your principles on this one.
So there are your alternatives. Incidentally, this should explain why
larger program files than yours can exist even though you've run into
the USL file limit -- the USL directory size is more a function of the
number of procedures you have than of your total code size.
You can take some comfort from the fact that your problems prove that
you have a good programming style; you must obviously use a lot of
relatively small procedures, which is usually a good thing -- until
you run into the USL directory size limit.
Oh, yes, there's another alternative -- buy a Spectrum, which doesn't
have this limitation in Native Mode. Good luck.
Q: Help! Sometimes the user friendly HP3000 can surely be unfriendly.
I really did not think that what I wanted to do was so far out of
line. But, as much as I try, I can't seem to get the HP to cooperate.
Here is the situation. I have a job that logs onto my A system (I'm
lazy, so I just named my computers A, B, C, D, and E). It then has to
remotely log on to my B system and remotely run a program (Robelle's
SUPRTOOL in this case). So far, everything works. Now, here is my
problem: how can the job tell if the remotely run program aborts?
I've tried the following:
!JOB MGR.OPR
!REMOTE HELLO MGR.OPR;DSLINE=B
!REMOTE RUN SUPRTOOL.PUB.ROBELLE
!IF JCW>=FATAL THEN
! TELLOP It aborted!
! EOJ
!ENDIF
!EOJ
Of course, since I did a REMOTE RUN, the remote JCW would be the one
that was set, not my local JCW. This way, my IF would never be
executed.
Then, I tried:
...
!REMOTE RUN ...
!REMOTE IF ...
! TELLOP It aborted!
! EOJ
!ENDIF
...
But, since I used a REMOTE IF, the TELLOP and the indented EOJ were
always executed.
Finally, I came up with a method that almost works:
...
!REMOTE HELLO MGR.OPR;DSLINE=B
!REMOTE REMOTE HELLO MGR.OPR;DSLINE=A
!REMOTE RUN ...
!REMOTE IF ...
!REMOTE REMOTE TELLOP It aborted!
!REMOTE ENDIF
...
Now, the REMOTE IF checks the REMOTE RUN and the REMOTE REMOTE TELLOP
tells the operator on A that the program on B aborted (since the
REMOTE REMOTE goes back to A, the local system). However, I can't
figure out how to cause the rest of the job to flush.
Is there a solution or will HP someday have to give us way to
interrogate JCWs that are on a remote computer?
A: Yes, there is a solution, and here it is.
Your problem is that you want to communicate information from your
remote machine (machine B) to your local machine (machine A). This
information is stored in a JCW, so the most logical way for you to
access it would be if HP let you access remote JCWs, e.g.:
...
!REMOTE HELLO MGR.OPR;DSLINE=B
!REMOTE RUN ...
!IF B:JCW >= FATAL THEN
! TELLOP ...
! EOJ
!ENDIF
...
Unfortunately, this is easier said than done. There's no way to
directly access JCWs across DSLINEs.
However, there is one sort of object that can be relatively easily
accessed across DSLINEs -- a file. The trick is to convert the
information stored in a JCW into "file form", access the file across
the DSLINE, and then, so to speak, convert the file back into a JCW.
Think of it as a sort of modem for JCWs!
What you should do is:
* Depending on the value of the JCW, you can build a file across a
DSLINE, e.g. by saying:
!PURGE FLAGFILE
!REMOTE RUN SUPRTOOL.PUB.ROBELLE
!REMOTE IF JCW<>0 THEN
!REMOTE COMMENT Build a file on machine A (the local one)
!REMOTE BUILD FLAGFILE;DEV=#
!REMOTE ENDIF
* Now, you either have (if JCW<>0) or don't have (if JCW=0) a file
called FLAGFILE on system A. At this point, you should "convert
the file back into a JCW":
!SETJCW CIERROR=0
!CONTINUE
!LISTF FLAGFILE;$NULL
!IF CIERROR=0 THEN
! COMMENT File exists, remote run must have aborted.
! PURGE FLAGFILE
! TELLOP It aborted!
! EOJ
!ENDIF
If FLAGFILE exists, then the !LISTF FLAGFILE;$NULL will leave the
CIERROR JCW set to 0; the !IF will see that CIERROR=0, send a
message to the console, and terminate the job. If FLAGFILE
doesn't exist (i.e. all went well), the !LISTF will fail and set
CIERROR to 907.
As you see, we've converted the information carried by the JCW (which
can't be accessed across DSLINEs) into information carried by the
presence or absence of a file (which CAN be accessed across DSLINEs).
Then, on the local machine, we've converted the presence/absence of a
file backed into a JCW using a !LISTF.
It's kinky, but it'll work.
Note that the :REMOTE BUILD X;DEV=# might not work in certain DS/NS
configurations (in particular, when the local machine is a DS/3000
machine and the remote is an NS/3000 machine). In this case, you can
just build the flag file on the remote machine, switch back to the
local machine, and then use DSCOPY to try to copy the flag file from
the remote to the local. You can then check the DSCOPY result to see
if the flag file existed.
The principle would still be the same -- use the existence or
non-existence of the flag file to record the failure or success of the
remote program.
Q: I have a small program that's supposed to do only a few things --
an FWRITE or two and a couple of DBGETs. I'd think that it should run
pretty fast; even if each FWRITE takes one disc I/O and each DBGET
takes as many as three (and I understand that shouldn't happen unless
my master dataset is really full), that shouldn't be more than about
10 I/Os. At 30 I/Os per second, that should take less than a second;
and, since my program is :ALLOCATEd, I imagine that it won't take too
long to load it.
Unfortunately, it seems that whenever the program is run, it takes at
least about five or ten seconds. This may not seem like much of a
problem, but I want to run it fairly often, at logon time and from
inside virtually every UDC that my users use (the program is supposed
to do some user activity logging). Why is it taking so long? Should I
change my files' blocking factors? My master datasets' capacities?
A: Only an FWRITE or two and a couple of DBGETs? Really? Isn't there
something you must be doing BEFORE doing those FREADs and DBGETs?
Every file you FWRITE must be FOPENed; every database you DBGET from
must be DBOPENed. An FWRITE might take 30 milliseconds if a disc I/O
is required and 3 milliseconds if a disc I/O is not required; an FOPEN
will take about 300-500 milliseconds. A DBGET might take 30
milliseconds (even if you have synonym chain trouble, you might get
away with one disc I/O); a DBOPEN can take up to 2 seconds!
I kicked all the other users off my MICRO/3000 XE, turned off caching
and did some tests. What I came up with was:
FOPEN of a vanilla disc file 0.3-0.5 milliseconds
FOPEN of a KSAM file about 1.2 seconds
DBOPEN of a database that about 2 seconds
nobody else has open
DBOPEN of a database that about 0.8 seconds
is opened by somebody else
(TurboIMAGE only)
First DBGET from a dataset in a about 0.4-0.5 seconds
database that nobody else has
open
First DBGET from a dataset in a about 50 milliseconds
database that somebody else has
open
If nobody else has your database open and you do DBGET from one
dataset and an FWRITE from one disc file, you're talking about 2.7 to
3 seconds in DBOPEN and FOPEN time alone! Of course, caching will cut
this time down by some unpredictable amount, but competition from
other users will drive it up by some (probably larger) amount.
Note that the time taken by the actually database or file read will be
about 0.06 seconds.
What can you do? For starters, you can minimize the number of files
you open. Possibly you open some files "just in case" and don't
actually use them; possibly you can restructure your application to
combine several files into one. Perhaps instead of writing to several
files, you can write one record to a message file that will then be
processed by a background job -- this job can then write the
appropriate records, having FOPENed the files once its life, rather
than once for each file.
If you use TurboIMAGE, you can take advantage of the fact that it
takes less time to DBOPEN a database that is already DBOPENed by
somebody else -- you might have a background job stream that simply
keeps the database open. Also, since each TurboIMAGE dataset is a
separate disc file, the first access to a dataset will force
TurboIMAGE to FOPEN the disc file (another 0.3 to 0.5 seconds) UNLESS
the dataset is already "globally opened". This will be the case if
somebody else has the database DBOPENed and has already accessed the
dataset, thus opening it.
It's even better if somebody has opened the dataset for WRITE access
(by doing a DBUPDATE, DBDELETE, or DBPUT to it), since if the dataset
is opened for READ access and you try to do a write to it, the system
will still have to re-FOPEN the dataset for you, not once but twice!
(Another 0.6 to 1 seconds.)
FOPEN and DBOPEN are slow simply because they have to do a lot of
work. An FOPEN has to:
* Find the file in the directory (about 3 or more I/Os on the
average).
* Read the file's file label from disc.
* Write the file label to disc to reflect the fact that the file is
open (various file label fields have to be changed).
* Possibly expand the opening process's stack to accommodate the
additional file table entries -- this might take another disc I/O
if there's not much room in main memory.
* If the file is buffered, possibly allocate an ACB (Access Control
Block) data segment to keep the file buffers -- this might again
require a disc I/O to flush something out of memory.
A DBOPEN has to do all this just to open the root file; then it has to
allocate various control data segments, read the root file tables into
them, update various root file fields, etc. This means a lot of CPU
processing and a lot of disc I/O.
FOPEN and DBOPENs take a long time (and FCLOSEs and DBCLOSEs take a
while, too). Though it's easier said than done, avoid them whenever
possible.
Q: Tell me one more time -- how can I erase a master dataset? I tried
the obvious thing (DBGET mode 2, DBDELETE, DBGET mode 2, DBDELETE,
etc.), and, sure enough, I found some records were left. I recall
hearing something some time about "migrating secondaries", but I've
forgotten it (and, to be frank, I never quite understood it in the
first place). What's the good word?
A: There are two things that need explaining: what the problem is and
what the solution is. First, the problem.
The simplest way of erasing a dataset is, as you said, to read it
serially and delete every record you read. For a detail dataset, it
will work just fine; but for a master, it won't, because WHENEVER YOU
DELETE A RECORD, IT'S POSSIBLE THAT A DIFFERENT RECORD WILL MIGRATE
INTO ITS PLACE. When you read the next record, you'll have already
skipped the migrated record, and thus you won't ever delete it.
Let's draw a little picture:
---------
| A |
---------
| B |
---------
| C |
---------
| D |
---------
| E |
---------
| ... |
---------
You read record A and delete it; you read record B and delete it; now,
you read record C and delete it. However (for reasons I'll explain
shortly), when you delete C, record F gets migrated into the place
that record C just occupied. Now, the file looks something like this:
---------
| empty |
---------
| empty |
---------
| F | <-- current read pointer
---------
| D |
---------
| E |
---------
| ... |
---------
Since you've just deleted record C, you get the next record (D),
delete it, get the next record (E), delete it, and so on; however,
you'll NEVER delete F, since it's managed to work its way into the are
you think you've already cleared.
Why did F migrate into C's slot? Well, master datasets are HASHED
datasets, in which a record's location is determined by the hash value
of its key. "C"'s hash value was 3, so C ended up in the 3rd record;
whenever we want to find C, we just hash it, get the value 3, and know
that we should look in the 3rd record.
However, it's possible that several keys may hash into the same value
-- in our case, C and F both hashed to 3. Clearly we can't put both of
them into the same record, so we put C into record 3, put F into some
other location, and put a pointer into record 3 that points to the
place where F is. F is now considered a "synonym" of C -- both C and F
are part of a "synonym chain", a linked list of records that hash to
the same value. C is the "primary" because it's actually put into the
location indicated by the hashing algorithm (record 3); F (and all
other things that hash to 3) are called "secondaries".
Note that F, being a secondary, takes longer to access. To DBGET the
record C, we just get the hash value (3), and look at record #3; to
DBGET the record F, we have to look at record #3, realize that it
isn't the one we're looking for, and then go look the next record in
the synonym chain. Accessing a primary takes only one I/O; accessing a
secondary can take longer.
Thus, when we DBDELETE C, IMAGE decides to be "smart" -- instead of
leaving F as a slow-to-access secondary, it MIGRATES F into record #3,
thus making F a primary. Now, whenever we try to DBGET F, you can get
to it in one I/O. Unfortunately, this "migration" prevents the record
from being deleted with our straightforward algorithm.
What can we do? Well, the trick is to realize that after you delete a
record, there may now be another record in the slot from which you've
just deleted. By looking at words 5 and 6 of the STATUS array that
DBGET returns you, you can tell if the record you've just deleted is a
primary, and if it is, how long the synonym chain is (including the
primary). Then, after you DBDELETE the primary, you can know not to do
another DBGET mode 2 (which will skip over the migrated secondary) but
instead do more DBDELETEs until you've deleted all the migrated
records.
Thus, you'd say:
WHILE not end of dataset DO
BEGIN
DBGET mode 2;
IF STATUS(words 5 and 6) = 0 THEN
<< not a primary, just delete it >>
DBDELETE
ELSE
FOR I:=1 UNTIL STATUS(words 5 and 6) DO
DBDELETE;
END;
If we run into a secondary, we delete it without any migration
concerns (since migration only happens when we delete a primary); if
it's a primary, we do not just one DBDELETE but rather as many
DBDELETEs as there are entries on the synonym chain (1 if there's only
the primary, 2 if there's one secondary, etc.). If there are 10
records on a synonym chain (one plus 9 secondaries), each of the
secondaries will migrate into the primary spot as the primary is
deleted; thus, it'll take us exactly 10 DBDELETEs to get rid of the
entire chain. (The doubleword stored in STATUS words 5 and 6 will in
this case be 10.)
So, there you go -- for each record you DBGET, you may have to do more
than one DBDELETE! I strongly suggest that you put a LOT of comments
in this piece of code when you write it.
Q: Every year, twice a year, I get really messed up by this whole
Daylight Savings Time thing. It's hard enough to change all my own
clocks and watches, but we also have to change the system times on our
dozen HP3000s.
Invariably, every year somebody forgets, and we don't realize what
happened until a whole bunch of data entry records have been
time-stamped incorrectly; even if we remembered to do everything on
Monday morning, the changeover happens on Sundays, so any user who's
working on Sunday will be in trouble.
Can't my computer do this for me? I understand that some UNIX systems
have these calculations built-in -- why can't the HP3000?
A: Computers can do everything! (Well, almost.) Here's what you can
do:
* First of all, you need to make sure that you have the contributed
unsupported CLKPROG utility on your system (I believe that it's
available in TELESUP) -- if you don't, get it quickly!
* Then, you have to be able to figure out WHEN the changeover will
take place so you can schedule a job to run then. You would most
likely want to figure this out from some nightly job you run
(e.g. your system backup).
* Finally, have a job that runs CLKPROG (which prompts for a date
and time), feeding it the correct adjusted date and time
automatically.
When does Daylight Savings Time come into effect? I called the U. S.
Government reference information number (I'm telling you this so
you'll blame them, not me, if this is wrong) and they said that we
"spring forward" one hour at 2 AM (which instantly becomes 3 AM) on
the first Sunday in April and we "fall back" at 2 AM (which instantly
becomes 1 AM) on the last Sunday in October.
Let's say that, every Friday, you do a backup that runs before
midnight. How can you tell if the upcoming Sunday is the first Sunday
in April? Well, the first Sunday in April has to fall between 1 April
and 7 April; if today is a Friday between 30 March and 5 April, then
we know that the next Sunday will be "spring forward".
Similarly, the last Sunday in October must be between 25 October and
31 October; thus, if today is a Friday between 23 October and 29
October, then the next Sunday will be "fall back".
So, your Friday night backup job should say:
!IF HPDAY=6 AND ((HPMONTH=3 AND HPDATE>=30 AND HPDATE<=31) OR
(HPMONTH=4 AND HPDATE>=1 AND HPDATE<=5)) THEN
! STREAM DLITEON;DAY=SUNDAY;AT=2:00
!ENDIF
!IF HPDAY=6 AND HPMONTH=10 AND HPDATE>=23 AND HPDATE<=29 THEN
! STREAM DLITEOFF;DAY=SUNDAY;AT=2:00
!ENDIF
(Note that we check that HPDAY is actually 6, i.e. that the job is
actually running on Friday, just in case the job is actually run on
some other day. This algorithm can easily be adapted to jobs that,
say, run on Saturday -- all that's necessary is that there be SOME job
that is guaranteed to run shortly before the changeover should take
place.)
Now, what about running CLKPROG? CLKPROG (at least my version,
A.00.01) prompts for the date, in MM/DD/YY format, and for the time --
how do you supply the new date and time to it? You'd like to be able
to say:
!JOB DLITEON,MANAGER.SYS;OUTCLASS=,1
!RUN CLKPROG.UTIL.SYS
SAME
3:00
!EOJ
to tell CLKPROG to keep the same date and to change the time to 3:00
AM; however, CLKPROG demands that you explicitly specify a new date,
even if it's the same as today's date.
What can you do? You have to use "MPE programming" techniques to build
a ;STDIN= file that you can then pass to CLKPROG. The idea is that
we'll somehow do a :SHOWTIME into a disc file, use EDITOR to translate
the :SHOWTIME output into CLKPROG input, and then feed this input to
CLKPROG.
Here's how it's done:
!JOB DLITEON,MANAGER.SYS;OUTCLASS=,1
!COMMENT Switches to Daylight Savings Time.
!IF NOT (HPDAY=1 AND HPMONTH=4 AND HPDATE>=1 AND HPDATE<=7) THEN
! TELLOP DLITEON streamed on the wrong day!
! EOJ
!ENDIF
!
!BUILD TIMEFILE;REC=-80,,F,ASCII;TEMP
!FILE TIMEFILE,OLDTEMP;SHR;GMULTI
!RUN FCOPY.PUB.SYS;INFO=":SHOWTIME";STDLIST=*TIMEFILE
!
!EDITOR
TEXT TIMEFILE
DELETE 1/3 << just FCOPY headers, ignore them >>
COPY 4 TO 5
CHANGE "SUN, APR ", "04/", 4
CHANGE ", 19", "/", 4
CHANGE 9/72, "", 4 << get rid of time >>
CHANGE 1/19, "", 5 << get rid of date >>
CHANGE " AM", "", 5
CHANGE " 2", " 3", 5 << increment hour >>
:PURGE TIMEFILE,TEMP
:FILE TIMEOUT=TIMEFILE;TEMP
KEEP *TIMEOUT
EXIT
!
!RUN CLKPROG.UTIL.SYS;STDIN=*TIMEFILE
!EOJ
There you go. Simple, isn't it? (Heh heh heh.) A very similar job
stream will switch back to non-Daylight Savings Time -- just change
the appropriate months, days, and hours. I imagine that minor
modifications might also make this work for other countries that have
similar schemes.
Note that the above would be somewhat simpler in MPE/XL, and even more
simple in VESOFT's MPEX or STREAMX.
Incidentally, you are quite correct that some UNIX systems do indeed
have this sort of thing built into the operating system. In 1987, in
fact, the vendors had to send out a patch to their users because the
U. S. Congress changed the changeover dates to lengthen the Daylight
Savings Time period. (The system could predict the time changes, but
no computer can predict what Congress will do!)
Incidentally, the U. S. Transportation Department estimates that the
1987 changeover (which, I believe, lengthened Daylight Savings Time by
about six or seven weeks) saved $28,000,000 in traffic accident costs
and avoided 1,500 injuries and 20 deaths. Believe it or not.
Q: The one thing that's always bothered me about the :REPLY command is
that it requires this completely arbitrary PIN value. My console
operator has to do a :RECALL, find the pin value, do the :REPLY -- why
can't he just say :REPLY 7 and have the tape request be satisfied? Or
(to avoid possibly :REPLYing to the wrong request), perhaps the
operator could somehow identify the request he's replying to, e.g.
:REPLY MYSTORE,7. This would be much simpler for my operators.
I thought that HP might do something about this with its user
interface improvements on the Spectrum, but no dice -- I've tried it
on my MPE/XL machine and the :REPLY command is the same as it's always
been.
Is there anything that can be done? I guess that somebody might write
a program that goes through the right system tables, but I don't know
how to do this and in any case don't have the time.
A: Going through the system tables would be a pretty drastic approach.
After all, the information you want (the pin) is readily available
using the :RECALL command -- the trick is capturing it and passing it
to the :REPLY command.
Well, you might ask, how am I going to "capture" the :RECALL output?
Unlike the :LISTF and :SHOWJOB commands, :RECALL does not have a "list
file" parameter -- all its output goes to the terminal. How can the
output be sent to a file?
Fortunately, the :RECALL output doesn't actually go to the terminal;
it goes to $STDLIST. If you can redirect $STDLIST -- as you can by
running a program with the ;STDLIST= parameter -- then the :RECALL
output will go to the ;STDLIST= file. Thus, you can say:
:PURGE MYLIST
:BUILD MYLIST;REC=-80,,F,ASCII;NOCCTL
:FILE MYLIST,OLD;SHR;MULTI
:RUN FCOPY.PUB.SYS;INFO=":RECALL";STDLIST=*MYLIST
Why do we run FCOPY here? We don't do it to copy any files -- rather,
we do it because FCOPY can execute as an MPE command any command
(prefixed by a ":") passed via ;INFO=. To use the ;STDLIST= parameter,
we have to run a program; to get the :RECALL output, we have to make
the program execute the :RECALL command -- rather than writing a
custom program that will do nothing but a :RECALL, we might just as
well use FCOPY.
OK, now there's a file called MYLIST that has the :RECALL output
inside it. (Actually, it also has two lines of overhead at the
beginning.) Now, we want to get the PIN from the :RECALL listing and
do a :REPLY to that PIN -- how can we do it?
We could write a program that opens MYLIST, reads it, finds the
correct record, and issues a :REPLY command (using the COMMAND
intrinsic). This would not be a trivial program (especially in COBOL);
furthermore, the moment a user-written program is introduced, that
substantially increases the management overhead involved. You'll have
to keep track of both the source file and the program file (in
addition to the command file or UDC that runs the program); if you
need to do the same thing on another machine, you'll have to transfer
at least two files, possibly all three. I prefer to have things as
self-contained and as simple as possible; whenever possible, I like to
avoid writing custom system programs.
But, you ask, how is it possible to solve the problem without writing
a third-generation-language program? Those of you who are familiar
with "MPE Programming" -- writing programs in the Command Interpreter
itself, using UDCs, job streams, and (in MPE/XL) command files and
variables -- might already have a pretty good idea of how this can be
done.
By the way, before we go any further -- the solutions that I mention
below are designed for MPE/XL. However, even if you're an MPE/V user,
they may still be interesting to you, since you can learn a bit about
MPE/XL from them. Furthermore, if you use VESOFT's MPEX/3000, you'll
be able to execute these command files from within it, since MPEX
emulates MPE/XL commands under MPE/V.
Our MYLIST file contains data that we want to read and use in a :REPLY
command. It would be nice if there were an :FREAD command that could
read a record from a file into an MPE/XL variable; however, there
isn't. There is an :INPUT command, but that only asks the user for
input, so we can't use it, can we?
Of course, we can. If we can redirect :RECALL's output to $STDLIST by
doing the :RECALL through a program with ;STDLIST= redirected, we can
also redirect :INPUT's input to MYLIST by doing the :INPUT through a
program with ;STDIN= redirected. In fact, if we only make MYLIST into
a message file, each :INPUT from the file will delete the record it
read, so we can (using a :WHILE loop) read the entire contents of the
file.
This is what our command file might look like:
PARM FILENAME, REPLYVALUE
PURGE MYLIST
BUILD MYLIST;REC=,,F,ASCII;NOCCTL;MSG
FILE MYLIST,OLD;SHR;GMULTI
RUN FCOPY.PUB.SYS;INFO=":RECALL";STDLIST=*MYLIST
WHILE FINFO('MYLIST',19)>0 DO
RUN FCOPY.PUB.SYS;INFO=":INPUT MYLINE";STDIN=MYLIST;STDLIST=$NULL
IF POS('"!FILENAME"',MYLINE)<>0 THEN
SETVAR MYLINE RHT(MYLINE,LEN(MYLINE)-POS("/",MYLINE))
SETVAR MYLINE RHT(MYLINE,LEN(MYLINE)-POS("/",MYLINE))
REPLY ![LFT(MYLINE,POS("/",MYLINE)-1)],!REPLYVALUE
ENDIF
ENDWHILE
It takes two parameters -- the name of the tape file for which the
reply was issued (in a message such as 'LDEV# FOR "MYTAPE" ON TAPE
(NUM)?', this would be MYTAPE) and the value to reply with (e.g. the
ldev). The way it works is as follows:
* The first four commands -- :PURGE, :BUILD, :FILE, and :RUN
FCOPY;INFO=":RECALL" -- send the :RECALL into MYLIST, a message
file.
* The "WHILE FINFO('MYLIST',19)>0" tells the CI to execute the
WHILE-loop commands while the number of records in MYLIST
(returned by FINFO option 19) is greater than 0, i.e. while we
haven't read all the records.
* The :RUN FCOPY;INFO=":INPUT MYLINE";STDIN=MYLIST executes the
:INPUT command from the MYLIST file. Note that we run FCOPY with
;STDLIST=$NULL so that the FCOPY headers and such junk aren't
printed.
* The :IF makes sure that this is indeed the right :REPLY request
-- the one that refers to the given filename; if it is, the two
SETVARs get rid of the text of MYLINE up to the second slash
(this will be the time and the job/session number), and the
![LFT(MYLINE,POS("/",MYLINE)-1] extracts just the PIN. As I'm
sure you've guessed (if you didn't know it already), POS and RHT
are string-handling functions -- in this case, we're using them
to parse the :REPLY command output.
So, there we have it. It could use a little bit of cleaning up; for
instance, I'd purge the file MYLIST and delete the variable MYLINE
after the command file is done; I'd check for the case where the reply
request we're looking for wasn't found and print an error message; I'd
also say
SETVAR OLDHPMSGFENCE HPMSGFENCE
SETVAR HPMSGFENCE 1
PURGE MYLIST
SETVAR HPMSGFENCE OLDHPMSGFENCE
instead of just saying "PURGE MYLIST" -- this will avoid the ugly
warning message in case the file MYLIST doesn't exist. ("SETVAR
HPMSGFENCE 1" means "print error messages but don't print any
warnings".)
Finally, one problem with this solution is that each :RUN FCOPY will
output a few blank lines and an "END OF PROGRAM" message -- you might
want to :ECHO some escape sequences that will get rid of this
undesirable output.
Had enough? Are you happy? Confused? Bored? Well, you're not off the
hook yet!
Take a look at the following command file:
PARM FILENAME, REPLYVALUE
ECHO ![CHR(27)+"J"]
RECALL
INPUT MYLINE;PROMPT=![CHR(27)+"A"+CHR(27)+"d"]
WHILE MYLINE<>"THE FOLLOWING REPLIES ARE PENDING:" AND &
MYLINE<>"NO REPLIES PENDING (CIWARN 3020)" AND &
POS('"!FILENAME"',MYLINE)=0 DO
INPUT MYLINE;PROMPT=![CHR(27)+"A"+CHR(27)+"A"+CHR(27)+"d"]
ENDWHILE
ECHO ![CHR(27)+"F"]
IF MYLINE="THE FOLLOWING REPLIES ARE PENDING:" OR &
MYLINE="NO REPLIES PENDING (CIWARN 3020)" THEN
ECHO Error: No such reply request!
ELSE
SETVAR MYLINE RHT(MYLINE,LEN(MYLINE)-POS("/",MYLINE))
SETVAR MYLINE RHT(MYLINE,LEN(MYLINE)-POS("/",MYLINE))
REPLY ![LFT(MYLINE,POS("/",MYLINE)-1)],!REPLYVALUE
ENDIF
What's going on here? Well, remember that an ASCII ESCAPE character is
character number 27; ![CHR(27)+"J"] means "escape J", which to HP
terminals means "clear the screen from the cursor to the end of the
screen". After we clear the remainder of the screen, we
* Do a :RECALL -- note that this :RECALL is not redirected
anywhere; it goes straight to your terminal.
* Do an :INPUT MYLINE;PROMPT=![CHR(27)+"A"+CHR(27)+"d"]
The escape-"A" means move the cursor up one line; the escape-"d"
means TRANSMIT THE LINE ON WHICH THE CURSOR IS LOCATED TO THE
COMPUTER! The escape-"d" tells the terminal to send the computer
the current line of text (nwhich is now the bottom line of the
:RECALL output) -- the :INPUT command dutifully picks up the line
and puts it into MYLINE.
* The :WHILE loop keeps doing these :INPUT commands until:
- MYLINE is equal to "THE FOLLOWING REPLIES ARE PENDING:" --
this would mean that we've read through all the reply
requests and haven't found the one we want;
- MYLINE is equal to "NO REPLIES PENDING (CIWARN 3020)";
- or, MYLINE contains the filename we're looking for, in which
case this is the reply request that we want.
* When we're done with the :WHILE loop, we either print an error
message indicating that the reply request wasn't found, or parse
the reply request to get its pin (just as we did in the previous
example).
Thus, this second approach does exactly what the first approach does,
but with three advantages:
* It doesn't require any :RUNs, so it can be done from break mode.
* It may be somewhat faster (again, because it doesn't :RUN any
programs).
* It doesn't output any pesky "END OF PROGRAM" messages.
As you see, there's more than one way to skin a :REPLY listing. Both
of them are good examples of how MPE/XL plus a little bit of ingenuity
can solve some pretty complicated problems. (The ingenuity on the
second example, incidentally, was a combination of mine and Gil
Milbauer's -- one of our technical support people at VESOFT. Thanks,
Gil.)
Q: I've been looking at the new ACD [Access Control Definition]
feature in V-delta-4 MIT and I have one question: where are these
things kept? Are they stored somewhere in the user labels? In the file
label? (If they're in the file label, where are they -- I've thought
that the file label was only 128 words long.)
A: Naturally, this is the very same question that I asked myself when
we first got V-delta-4. There's almost no room left in the 128-word
file label, certainly not enough to keep the ACD pairs (of which there
can be up to 20, each requiring at least 9 words of storage).
What I did to figure this out was relatively straightforward. I built
a file without an ACD and did a :LISTF ,-1 (show file label) of it;
then I added an ACD, did another :LISTF ,-1, and compared.
The differences that I saw were in words 120, 121, and 122. Word 122
was a "1" (I'll get to it shortly), and words 120 and 121 looked
suspiciously like a two-word disc address (just like the disc
addresses in the extent map -- the first 8 bits were the volume table
index and the last 24 bits were the sector address).
The next step was to run DISKED5 and look at the sector pointed to by
the label. Sure enough, there was the ACD in all its glory -- 6 words
of overhead information followed by 9 words for each pair (4 words for
the user name, 4 words for the account name, and 1 word for the access
mask). In a way, this place where the ACD is kept is a
"pseudo-extent", just like the file's 32 data extents -- a chunk of
disc space pointed to by the file label. In fact, "pseudo-extent" is
exactly what HP itself calls it.
Of course, this pseudo-extent is not the same size as the data
extents; it is 1 or 2 sectors long, depending on the ACD size. (The
number of sectors in the pseudo-extent is kept in word 122.) If you
build a file with 13 or less pairs in its ACD, only 1 sector is
allocated (since 6+9*13 = 123 is less than 128); if you then add a
pair, the old 1-sector pseudo-extent will be deallocated and a new
2-sector pseudo-extent will be allocated in its place.
Because of this, I'm quite surprised that HP limited the ACD to 20
pairs. Certainly there's room in even a 2-sector pseudo-extent for 27,
and there's no reason that I can see why the pseudo-extent can't grow
to 3 or more sectors. The best guess that I heard about this is that
somewhere they might try to read the entire ACD onto the stack and
they therefore don't want it to be too large. I'm not sure about this,
but I suspect that the 20-pair restriction might prove bothersome
under some circumstances.
The other thing that you should keep in mind with regard to ACDs is
that -- as of press time -- both EDITOR and TDP (and many other
editors) throw away ACD's of files they /KEEP over. Since an ACD is
attached to the file label, when the file is purged and re-built
(which is what happens during a /KEEP), the ACD goes away. One can use
the HPACDPUT intrinsic to explicitly copy the ACD from the old file to
the new, but EDITOR and TDP don't do this.
I had a lot of fun with this when I decided to change our "MPEX hook"
feature to force EDITOR and TDP (and many other editors) to preserve
these ACDs, even though they were never designed to! I patched the
programs' calls to the FCLOSE intrinsic so that when they did an
FCLOSE with disposition 4 (purge file) the ACD of the purged file
would be saved and when they then did an FCLOSE with disposition 1
(save file) the saved ACD would be attached to the new file (if it had
the same name as the just-purged file).
Now our hooked EDITOR, hooked TDP, hooked QUAD, etc. preserve ACDs of
files that you /KEEP over. Some time I might write something about how
these "hooks" are done -- how programs can be patched to do things
they were never intended to do, like preserve ACDs, have a multi-line
REDO command history, save files across account boundaries (if you
have SM), and so on. I find it to be a very interesting and powerful
mechanism.
Q: I've been doing some stuff on the Spectrum in compatibility mode --
their new debugger is great! It's so much better and easier to use
than the Classic Debugger in virtually all respects -- setting
breakpoints by procedure name (rather than octal address), symbolic
tracebacks, windows, etc.
Note that I said VIRTUALLY all respects. One thing that I can't seem
to get the system to do is to drop me into debug when my program
aborts (using the :SETDUMP command). If I do a :SETDUMP, I do get a
nifty symbolic traceback (which tells me exactly where the abort
occurred), but I can't get into debug to look at my variables,
procedure parameters, and so on -- all the stuff that can tell me why
the program aborted. I do a :HELP SETDUMP and it tells me that "if the
process is interactive, it will subsequently enter the system debugger
to wait for further commands", but that doesn't seem to happen. What
can I do?
A: Well, you're right. :HELP SETDUMP tells you that a :SETDUMP will
get you into the debugger in case of a program abort; the MPE Commands
Manual says the same thing. But the System Debug Reference Manual
says:
"If the process is being run from a session, then after the
specified command string has been executed, DEBUG will stop to
accept interactive commands with I/O performed at the user
terminal, CONTINGENT UPON THE FOLLOWING REQUIREMENTS:
* The abort did not occur while in system code
* The process entered the abort code via a native mode
interrupt.
NOTE: CM programs will usually fail these tests."
Because of the way MPE/XL executes CM programs, it seems that any CM
program abort is always "in system code", and a program abort -- even
when you've done a :SETDUMP -- will never (well, almost never) drop
you into the debugger.
Great -- what now? Well, it turns out that there IS something you can
do (though only if you have PM capability). Instead of having the
:SETDUMP command get you into debug AFTER the abort occurs, you can
get into debug WHILE the aborting is in progress. Just say:
:RUN MYPROG;DEBUG
cmdebug > NM; B trap_handler,,,CM; E
This sets a break point at the system SL procedure "trap_handler"
which is called (to the best of my knowledge) whenever the program is
aborting. When "trap_handler" is triggered, the breakpoint will be
hit, and you'll be in DEBUG, able to look at anything you want.
Note that we didn't just say "B trap_handler" but "B
trap_handler,,,CM". The fourth parameter of the B command is the list
of commands to be executed whenever the breakpoint is hit; in this
case, we tell DEBUG to do a "CM" to get us into compatibility mode
DEBUG rather than into native mode DEBUG.
If you do this, you don't need to do a :SETDUMP anymore. The bad part
is that, unlike :SETDUMP, you can't do it once for your session, but
rather have to do it every time you run your program. If you really
want to, you can have your program call the HPDEBUG intrinsic with the
"NM;B trap_handler,,,CM;E" command as the first thing it does.
However, since HPDEBUG is a Native Mode intrinsic, your CM program
would have to call it via Switch (with the HPSWTONMNAME intrinsic)
rather than call it directly.
One cautionary note: It's always risky to rely on system internal
procedures like "trap_handler", since they can change at any time.
I've tested this on MPE/XL Version A.10.01 (colloquially known as
"MPE/XL 1.1") -- I make no promises beyond that.
Q: One of my programs has a bug that only seems to manifest itself in
a particular (very long) job stream. Whenever I try to duplicate it
online, I can't; however, if it's running in batch, I can't use the
debugger. Finally I manage to learn how to use DEBUG (fortunately,
it's a lot more powerful on MPE/XL than on MPE/V), and I can't apply
to it to the one problem I need to fix!
Is there anything I can do?
A: Yes, there is. Fortunately, among the many wonderful improvements
implemented in DEBUG/XL is the ability to debug batch programs on the
console. If you go into DEBUG and say
ENV JOB_DEBUG = TRUE
then whenever a job calls DEBUG (usually as a result of a :RUN ;DEBUG
or of a Native Mode program abort following a :SETDUMP command), the
debugger will start executing on the system console on behalf of the
job. You can do anything in this mode that you normally would
(remembering that you're actually debugging the job, not whatever is
running on the console at the time); you can type "EXIT" to leave the
debugger and resume the job.
But, you ask, what about the session that I already have running on
the console? Excellent question! The session keeps running -- keeps
sending whatever output it has to the console (where it will come out
mixed in with the debugger output) -- even worse, keeps prompting you
for input on the console!
So what can you do? You must somehow suspend your console session so
that it will no longer try to use the terminal while the debugger's
using it. My recommendation is to say
BUILD MSGFILE;MSG
RUN FCOPY.PUB.SYS;STDIN=MSGFILE;STDLIST=$NULL
as soon as the DEBUG header appears on the console. You must be sure
that you enter this in response to your console session's prompt
instead of in response to the "nmdebug> " prompt. (These two prompts
will more or less alternate until you actually manage two execute the
BUILD and the RUN.) Once you manage to get these two commands in, your
console session will be waiting on the message file wait, and you can
go ahead using the debugger.
When you exit the debugger, you'll be back to your hung console
session. To unhang it, just hit [BREAK] and say
FILE MSGFILE,OLD;DEL
LISTF MSGFILE;*MSGFILE
RESUME
The :LISTF command will write a few records into MSGFILE, which will
(when the :RESUME is done) wake up FCOPY and return control to
wherever your console session was before.
So, there you go. A little bit of work, but it's worth it -- you can
now debug your program in exactly the environment in which it fails.
A couple of other things worth mentioning:
* If your program is aborting with a BOUNDS VIOLATION, INTEGER
OVERFLOW, or some such error, you should probably do a !SETDUMP
command in your job before !RUNing your program -- this makes
sure that DEBUG will be called when the program aborts.
This will work quite well for MPE/XL Native Mode programs, but
not for Compatibility Mode programs; even if you do a !SETDUMP,
CM programs will usually NOT drop you into the debugger at abort
time (although you can still !RUN them with ;DEBUG, set
breakpoints, etc.).
However, if you say in your job stream
!RUN CMPROG;DEBUG
and then enter at the console (when the debugger will be invoked)
B trap_handler,,,CM
then any program abort WILL call DEBUG (whether or not you do a
!SETDUMP) -- this is because "trap_handler" is the system
procedure that's called at program abort time, and by setting a
breakpoint at it, you make sure that DEBUG will be called
whenever "trap_handler" is called.
* If you execute MPE commands from within DEBUG-on-the-console-
running-on-behalf-of-your-job (using the : command), these
commands will also be executed on behalf of your job (not of the
console session). HOWEVER, this means that their output will be
sent to your job's $STDLIST, where you won't be able to see it
until the job finishes!
Thus, if you want to do a :LISTEQ, a :LISTFTEMP, a :SHOWVAR, or
whatever to see what's going on in the job, you're in trouble.
Your best bet in this case would probably be to send the
command's output to a disc file and then look at this file from
another terminal.
On the other hand, remember that you can use this feature to
:BUILD or :PURGE job temporary files, set :FILE equations, etc.
Q: From within our COBOL programs we sometimes get these
nasty-looking "Illegal ASCII digit" messages; the program then
continues, but the results are usually meaningless.
Question #1: How can we tune the system to abort the program after
printing the usual "Illegal ASCII digit" stuff?
Question #2: As we have bought third-party programs (e.g. AR/AP) it
might happen that those programs abort now, which would not be
desirable. Is it possible to move the modified trap procedure
(COBOLTRAP?) to a group SL so that we can run the programs which we
want to abort with ;LIB=G or ;LIB=P?
Question #3: Is it possible to customize the procedure depending on
whether the error occurs in a session or in a job? All of our
interactive programs use VPLUS, so it is not very amusing to watch
the error message running through the fields. Most preferably the
program should get back an error along with the source- and
programcounter- addresses. Also, we'd like to see block mode turned
off and then have the usual message displayed with the final abort.
A: Good questions. Let's see if I can give you some good answers.
First of all, your questions seem to imply that you're looking for a
global sort of modification, perhaps a patch to SL.PUB.SYS.
Actually, this is possible, but I would not advise it. I try to
avoid operating system patches as much as possible -- they give HP
understandable concerns about supporting your systems, they're not at
all trivial to install, and they may have to be re-developed for
every new version of the operation system. Although several vendors
have had very good experience with products that patch MPE, I would
recommend that ordinary users not do it themselves (the vendors have
the time and money needed to support this kind of thing -- most users
don't).
Fortunately, there's a non-OS-patch solution that should take care of
(at least) your questions #1 and #2. Actually, there are two
solutions -- a simple one if you're using an MPE/XL system and a
somewhat more complicated one if you're using MPE/V.
On MPE/XL, the COBOL II/XL compiler DOES NOT (by default) CHECK FOR
ILLEGAL ASCII/DECIMAL DIGITS! In other words, by default, any such
errors will be silently ignored. If, however, you compile your
program with the $CONTROL VALIDATE compiler option, illegal
ASCII/decimal digits will cause the program to ABORT (rather than
print a warning, do a fix-up, and continue).
You can, however (if you use the right $CONTROL options), configure
the exact behavior in case of this error using an MPE/XL variable
called COBRUNTIME. It is supposed to be a string of six letters;
each letter indicates what is to be done in case of one of six kinds
of error, the possible values for each letter being:
A = abort
I = ignore
C = comment (message only)
D = DEBUG (go into DEBUG in case of error)
M = message and fixup
N = no message but fixup
The letter assignments are:
letter 1 governs behavior in case of ILLEGAL DECIMAL OR ASCII DIGIT;
letter 2 pertains to RANGE ERRORS;
letter 3 pertains to DIVIDES BY ZERO OR EXPONENT OVERFLOWS;
letter 4 pertains to INVALID GOTOS;
letter 5 pertains to ILLEGAL ADDRESS ALIGNMENT;
letter 6 pertains to PARAGRAPH STACK OVERFLOW.
Letter 1 is in effect only if you use $CONTROL VALIDATE; letters 2
and 6 are in effect only if you use $CONTROL BOUNDS; letter 5 is in
effect only if you use $CONTROL BOUNDS and either $CONTROL
LINKALIGNED or $CONTROL LINKALIGNED16.
Thus, if you say
:SETVAR COBRUNTIME "AIDDDD"
this tells the COBOL II/XL run-time library to abort in case of
illegal decimal or ASCII digit, ignore range errors, and go into
DEBUG for any of the other four conditions. (You might want to put
this :SETVAR into some OPTION LOGON UDC so that it is always in
effect).
The solution for Classic (non-Spectrum) machines is not as easy.
How does COBOL detect, report, and try to fix up illegal ASCII digits
and similar errors? Well, the HP3000's normal tendency in case it
finds this kind of error would be to abort the program (just as it
normally would for a stack overflow, a bounds violation, a divide by
zero, etc.). However, COBOL decided to do things differently, so it
took advantage of the XARITRAP intrinsic.
XARITRAP lets you indicate that when a particular arithmetic error
occurs, the program should not be aborted but instead a particular
"trap procedure" should be called. XARITRAP also lets you specify
exactly which arithmetic traps are to be re-routed through the trap
procedure; that way, your program might, for instance, trap integer
overflows but not divides by zero.
The first thing that a COBOL program does when it's run (well, almost
the first thing) is call the system SL COBOLTRAP procedure. This
COBOLTRAP procedure calls the XARITRAP intrinsic, telling it that all
integer and decimal arithmetic errors should be passed to another
system SL procedure called C'TRAP. C'TRAP is the one that prints the
error message, tries the fix-up, and also handles other errors that
you don't even see (for instance, integer overflows and divides by
zero, which it silently ignores).
The trick is that when COBOLTRAP calls XARITRAP, it passes it a
"mask" value of octal %23422; if you look at the XARITRAP
documentation in the Intrinsics Manual, you'll find that this traps
Integer Divide By Zero, Integer Overflow, and all the decimal
arithmetic errors (including your Illegal ASCII Digit error).
Now, as I said before, from MPE's point of view it's only natural for
any one of these arithmetic errors to cause your program to abort.
COBOLTRAP changes this default assumption; all you need to do is to
revert to the original state by unsetting the trap. One way of doing
this would simply be:
77 DUMMY PIC S9(4) COMP.
...
<< near the start of your program, do the following: >>
CALL "XARITRAP" USING 0, 0, DUMMY, DUMMY.
This simply turns off ALL arithmetic traps -- whenever any decimal or
integer arithmetic error occurs, the program is aborted.
The problem with this is that this disables ALL arithmetic traps,
including integer divides by zero and integer overflows. This is a
good deal broader than your original request of causing aborts only
on Illegal ASCII Digits; it's possible that your programs rely on the
trapping of integer overflows, integer divides by zero, and other
conditions. (I've even heard rumors that the code generated by the
COBOL compiler ALWAYS relies on integer overflows being trapped, but
I can't vouch for the truth of this.)
So what can you do? Well, the next step would be to do something
like this:
77 DUMMY PIC S9(4) COMP.
77 OLD-MASK PIC S9(4) COMP.
77 OLD-PLABEL PIC S9(4) COMP.
...
<< near the start of your program, do the following: >>
CALL INTRINSIC "XARITRAP" USING 0, 0, OLD-MASK, OLD-PLABEL.
COMPUTE OLD-MASK = OLD-MASK - (OLD-MASK / 256) * 256.
CALL INTRINSIC "XARITRAP" USING OLD-MASK, OLD-PLABEL, DUMMY, DUMMY.
What's going on here?
* We first call XARITRAP to disable all the traps; however, the
real reason we do this is to get back the OLD mask and the OLD
trap procedure address (PLABEL).
* Then, we use the COMPUTE statement to eliminate the high-order 8
bits of the mask -- these refer to all the decimal traps; I
presume that you'd like to see ALL of these cause aborts.
* Finally, we call XARITRAP again, passing the new mask and the
old plabel (to re-enable the trap procedure that we've just
disabled).
This way, you make the decimal traps cause aborts while retaining the
normal behavior for integer overflows and integer divides by zero.
Does this make sense? Did you put those lines into your program?
Are you happy? Well, you shouldn't be! Unfortunately, despite all
my reasonable arguments, the above code will do exactly what the
previous example did -- it will disable ALL arithmetic traps, not
just the decimal ones! Why?
Well, the last wrinkle that you have to iron out is caused by a
little check that the XARITRAP procedure does. If you look up
XARITRAP in your Intrinsics Manual, you'll see the following lines:
"If, when, a trap procedure is being enabled, the code of the
caller is ... nonprivileged in PROG, GSL, or PSL, plabel must be
nonprivileged in PROG, GSL, or PSL."
Makes sense to you? Of course it does -- clear as day! Actually,
what this means is that when you enable a trap from your program
(which you're doing with the second XARITRAP), you can't tell MPE to
trap to a procedure in the system SL (which is what we're telling it
to do by passing the plabel we got back from the first XARITRAP call
-- the plabel of the system SL C'TRAP procedure).
I think that this is actually a security feature (otherwise you might
be able to trap to some internal MPE procedure which might then crash
the system and generally cause unpleasantness); however, this means
that those three lines -- the first CALL "XARITRAP", the COMPUTE, and
the CALL "XARITRAP" actually have to go into a procedure that is kept
in the system SL. Such a procedure might be:
$CONTROL DYNAMIC, NOSOURCE, USLINIT
IDENTIFICATION DIVISION.
PROGRAM-ID. VENODECTRAP.
AUTHOR. Eugene Volokh of VESOFT, Inc.
DATE-WRITTEN. 6 February 1988.
ENVIRONMENT DIVISION.
CONFIGURATION SECTION.
DATA DIVISION.
WORKING-STORAGE SECTION.
77 MASK PIC S9(4) COMP.
77 PLABEL PIC S9(4) COMP.
PROCEDURE DIVISION.
COBOLERROR-PARA.
CALL INTRINSIC "XARITRAP" USING 0, 0, MASK, PLABEL.
COMPUTE MASK = MASK - (MASK / 256) * 256.
CALL INTRINSIC "XARITRAP" USING MASK, PLABEL, MASK, PLABEL.
GOBACK.
Now, to turn off decimal traps, your program should just say:
CALL "VENODECTRAP".
Since VENODECTRAP will be in the system SL, the second XARITRAP call
(which re-enables the system SL C'TRAP procedure) will work.
A few more points remain. For one, what if you ONLY want to abort in
case of an Illegal ASCII Digit error and don't want to change the
behavior in case of the other errors? What you need to do for that
is turn off the bit in the MASK variable that corresponds to the
Illegal ASCII Digit error (which is bit 6); you can do this by
saying:
COMPUTE MASK = MASK - 512.
instead of the original "COMPUTE MASK = ..." statement. This should
always work as long as the bit is actually set to begin with;
otherwise, the subtraction of 512 will have rather unusual results
(it'll probably ENABLE trapping of Illegal ASCII Digit errors but
disable the trapping of Illegal Decimal Digit errors). This could be
a problem if your program calls VENODECTRAP twice.
Alternatively, you could instead say
MOVE %22422 TO MASK.
Normally, as I mentioned, the COBOL library sets the mask to be
%23422; %22422 is the same but with bit 6 turned off. The only
trouble with this approach is that if the COBOL library is ever
changed to trap some other things, the %22422 could inadvertently
unset the traps that the new library procedures set.
Actually, we discussed doing bit arithmetic in COBOL a few months ago
in this column; if you want to, you can find this discussion and use
the algorithms listed there to unset bit 6. It was a long story,
however, and I don't want to get further into it just now.
Finally, your question #3 was: how do I avoid having the abort
displays appear in my V/3000 forms fields? This is a much more
difficult question. It would be nice if one could set one trap
procedure (the default C'TRAP) for the traps that you want COBOL to
handle normally and a different trap procedure for those for which
you want to abort; then, your own trap procedure could get out of
block mode before aborting.
Unfortunately, MPE doesn't let you have two different arithmetic trap
routines in effect at once (even if they apply to different error
conditions). One thing that I can recommend is a package called
ATLAS/3000 from
STR Software Co. P. O. Box 12506 Arlington, VA 22209 (703) 689-2525
fax (703) 689-4632
ATLAS/3000 traps all sorts of errors (COBOL errors, IMAGE errors, job
errors, file system errors, etc.), logs them, reports them to the
console, and so on. I'm not sure that it's exactly what you're
asking for, but it could quite possibly help you out in this area.
One last point: by disabling the normal COBOL trapping for Illegal
ASCII digits, you're also losing the SOURCE ADDRESS and SOURCE
displays that the trap routine normally gives you; these could
sometimes help you identify which piece of data is bad. You also
lose the STACK DISPLAY that the cobol library prints for you (which
shows you not only where in your code the abort occurred but also a
traceback of which procedures called that piece of code); however,
you can re-enable this stack display by saying
:SETDUMP
before running your program.
Q: I have what you might think to be a strange question -- where is
the Command Interpreter?
On my MPE/XL machine, I see a program file called CI.PUB.SYS, so I
guess that that's the Command Interpreter program (the one that
implements :RUN, :PURGE, etc.) -- however, I couldn't find such a
program on my MPE/V computer.
:EDITOR, I know, is implemented through EDITOR.PUB.SYS; :FCOPY is
FCOPY.PUB.SYS; all the compilers have their own program files. Where
is the CI kept?
A: This is a very interesting question -- one that bothered me for a
while several years ago. The answer is somewhat surprising.
Virtually all of the operating system on MPE/V is kept in the system
SL (SL.PUB.SYS). This includes the file system, the memory manager,
etc.; it also includes the Command Interpreter. But, you may say,
each job or session has a Command Interpreter process, and each
process must have a program file attached to it. Nope. Each of YOUR
processes must have a program file attached to it; MPE itself faces
no such restrictions.
All that a process really has to have is a data segment (which MPE
builds for each CI process when you sign on) and code segments;
internally, there's no reason why those code segments can't be in the
system SL. MPE just uses an internal CREATEPROCESS-like procedure
that creates a process given not a program file name but an address
(plabel) of a system SL procedure; no program file needed.
On MPE/XL, the situation is somewhat different (as you've noticed);
there's a program file called CI.PUB.SYS that is actually what is run
for each CI process. However, if you do a :LISTF CI.PUB.SYS,2,
you'll find that it's actually quite small (186 records on my MPE/XL
1.1 system); since Native Mode program files usually take a lot more
records than MPE/V program files, those 186 records can't really do a
lot of work. CI.PUB.SYS is really just a shell that outputs the
prompt, reads the input, and executes it by calling MPE/XL system
library procedures. MPE/XL library procedures (including the ones
that CI.PUB.SYS calls, either directly, or indirectly) are kept in
the files NL.PUB.SYS, XL.PUB.SYS, and SL.PUB.SYS.
Thus, in both MPE/V and MPE/XL, virtually all of the smarts behind
command execution is kept in the system library (or libraries).
Notable exceptions include all the "process-handling" commands, like
EDITOR, FCOPY, STORE/RESTORE (which use the program STORE.PUB.SYS),
and even PREP (which actually goes through a program called
SEGPROC.PUB.SYS). However, :FILE, :PURGE, :BUILD, and so on, are all
handled by SL, XL, or NL code.
Q: A couple of months ago I read in this column that you can speed up
access to databases by having somebody else open them first. Is
there some easy way that I can take advantage of this without having
to write a special program? I guess what I'd have to do is have a
job that DBOPENs the database and then suspends with the database
open, but how can this be easily done?
A: Indeed, if a TurboIMAGE database is already DBOPENed by somebody
then a subsequent DBOPEN will be faster than it would be if it were
the first DBOPEN. On my Micro XE, a DBOPEN of a database that nobody
else has open took about 2 seconds; a DBOPEN of a database that was
already opened by somebody else took only about 0.8 seconds. The
first DBGET/DBPUT/DBUPDATE against a dataset were also faster if the
database was already open (for the DBGET, about 50 milliseconds vs.
about 400-500 milliseconds).
How can you take advantage of this? Well, if the database is
constantly in use (e.g. it's opened by each of the many users that is
accessing your application system), then the time savings comes
automatically -- the first user will spend 2 seconds on the DBOPEN
and all subsequent users will spend 0.8 seconds, as long as the
database is open by at least one other user.
The problem arises when your database is frequently opened but is
left open for only a short time; for instance, if it is used by some
short program that only needs to do a few database transactions
before it terminates. Then, you might have hundreds of DBOPENs every
day, the great majority of which happen when the database is NOT
already open. You'd like to have one job hanging out there keeping
the database open so that all subsequent DBOPENs will find the
database already opened by that job.
How can this be done without writing a special custom program? (I
agree that you should try to avoid writing custom programs whenever
possible -- if you wrote a special program, you'd have to keep track
of three files [the job stream, the program, and the source] rather
than just the job stream.) Well, let's take this one step at a time.
Having a job stream that opens your database is easy -- just say
!JOB OPENBASE,...
!RUN QUERY.PUB.SYS
BASE=MYBASE
<<password>>
1 <<the mode>>
Unfortunately, as soon as this job opens the database, it'll start to
execute all the subsequent QUERY commands; eventually (because of an
>EXIT or an end-of-file) QUERY will terminate and the database will
get closed. It would be nice if QUERY had some sort of >PAUSE
command (or, to be precise, a >PAUSE FOREVER command), but it
doesn't.
Or does it? What are the mechanisms that MPE gives you for
suspending a process? Can we use any of them from within QUERY?
Well, there's obviously the PAUSE intrinsic, which pauses for a given
amount of time. It's not quite what we want (since we want to pause
indefinitely), but more importantly there is really no way of calling
PAUSE from within QUERY (unless your QUERY is hooked with VESOFT's
MPEX). So much for that idea. There are also a few other intrinsics
-- for instance, SUSPEND and PRINTOPREPLY -- that suspend a process,
but they're not callable from QUERY either. You can't call
intrinsics from QUERY; you can't even do MPE commands (though in any
event there aren't any MPE commands that suspend a process).
One other thing, however, comes to mind -- message files. A read
against an empty message file is a good way to suspend a process;
however, QUERY doesn't have an ">FOPEN" or an ">FREAD" command,
either, does it?
Well, in a way it does. QUERY's >XEQ command lets you execute QUERY
commands from an external MPE file, and to do that it has to FOPEN it
and FREAD from it. You might therefore say:
!JOB OPENBASE,...
!BUILD TEMPMSG;MSG;TEMP;REC=,,,ASCII
!RUN QUERY.PUB.SYS
BASE=MYBASE
<<password>>
1 <<the mode>>
XEQ TEMPMSG
EXIT
!EOJ
The XEQ will start reading from this temporary message file, see that
it's empty, and hang, waiting for a record to be written to this
file. Since the file is a temporary file, nobody else will be able
to write to it, so the job will remain suspended until it's aborted.
There are a few other alternatives along the same lines. For one,
you could make the message file permanent -- that way, you can stop
the job by just writing a record to this file. Why would you want do
this? Because you may want to have your backup job automatically
abort the OPENBASE job so that the database can get backed up.
If the message file were permanent, your backup job could then just
say:
!FCOPY FROM;TO=OPENMSG.JOB.SYS
<< just a blank line -- any text will do >>
...
The one thing that you'd have to watch out for is the security on the
OPENMSG.JOB.SYS file -- you wouldn't want to have just anybody be
able to write to this file because any commands that are written to
the OPENMSG file will be executed by the QUERY in the OPENBASE job
stream. (Remember the >XEQ command.)
If you don't use the permanent message file approach, you can still
have the background job abort the OPENBASE job by saying
!ABORTJOB LOCKBASE,user.account
However, for this, you'd either have to have :JOBSECURITY LOW and
have your backup job log on with SM capability, or have the :ABORTJOB
command be globally allowed (unless you have the contributed ALLOWME
program or SECURITY/3000's $ALLOW feature).
Another alternative is to >XEQ a file that requires a :REPLY rather
than a message file, e.g.
!JOB OPENBASE,...
!FILE NOREPLY;DEV=TAPE
!RUN QUERY.PUB.SYS
BASE=MYBASE
<<password>>
1 <<the mode>>
XEQ *NOREPLY
EXIT
!EOJ
The trouble with this solution is that the console operator might
then accidentally :REPLY to the "NOREPLY" request. (Also, the job
wouldn't quite work if you had an auto-:REPLY tape drive!)
In any event, though, one of the above solutions might be the thing
for you. It's not going to buy you too much time (it only saves time
for DBOPENs and the first DBGETs/DBPUTs/DBUPDATEs on each dataset),
it isn't necessary if the database is already likely to be opened all
the time, and it won't work on pre-TurboIMAGE or MPE/XL systems.
However, in some cases it could be a non-trivial (and cheap!)
performance improvement.
One note to keep in mind (if you really look carefully at the way
IMAGE behaves): the BASE= in the job stream will only open the
database root file -- the individual datasets will remain closed.
HOWEVER, once the keep-the-database-open job starts, any open of any
dataset by any user will leave the dataset open for all the others;
even when the user closes the database, the datasets will still
remain open. If the user only reads the dataset (and doesn't write
to it), the dataset will only be opened for read access; however,
once any user tries to write to the dataset, the dataset will be
re-opened for both read and write access.
The upshot of this is that the database will start out with only the
root file opened, and then (as users start accessing datasets in it)
will slowly have more and more of its datasets opened. After each
dataset has been accessed (especially written to) at least once, no
more opens will be necessary until the last user of the database
(most likely the keep-open job itself) closes it.
Q: I recently tried to develop a way to periodically check my system
disk usage (free, used, and lost). I based it on information that
VESOFT once supplied in one of their articles.
My procedure was as follows:
1. Do a :REPORT XXXXXXXX.@ into a file to determine total disk
usage by account.
2. Subtotal the list (in my case, using QUIZ) to get a system disk
space usage total.
3. Run FREE5.PUB.SYS with ;STDLIST= redirected to a disk file.
4. Use QUIZ to combine the two output files and convert the totals
to Megabytes.
This way, I can show the total free space and total used space on one
report for easy examination. I can also add these two numbers, compare
the sum to the total disk capacity, and thus determine the 'lost' disk
space.
My question is in regard to the lost disk space. The first time I ran
the job, the total lost disk space came out to be approximately 19
Megabytes. After doing a Coolstart with 'Recover Lost disk Space', the
job again showed a total lost disk space of 19 Megabytes. Didn't the
Recover Lost disk Space save any space? Could there be something I'm
overlooking?
A: Unfortunately, there is. Paradoxical as it may seem, the sum of the
total used space and the total free space is NOT supposed to be the
same as the total disk space on the system; or, to be precise, there
is more "used space" on your system than just what the :REPORT command
shows you.
The :REPORT command shows you the total space occupied by PERMANENT
disk FILES. However, other objects on the system also use disk space:
- TEMPORARY FILES created by various jobs and sessions;
- "NEW" FILES, i.e. disk files that are opened by various processes
but not saved as either permanent or temporary;
- SPOOL FILES, both output and input (input spool files are
typically the $STDINs of jobs);
- THE SYSTEM DIRECTORY;
- VIRTUAL MEMORY;
- and various other objects, mostly very small ones (such as disk
labels, defective tracks, etc.).
Your job stream, for instance, certainly has a $STDLIST spool file and
a $STDIN file (both of which use disk space), and might use temporary
files (for instance, for the output of the :REPORT and FREE5); also,
if you weren't the only user on the system, any of the other jobs or
sessions might have had temporary files, new files, or spool files.
In other words, to get really precise "lost space" results, you ought
to:
* Change your job stream to take into account input and output
spool file space (available from the :SHOWIN JOB=@;SP and
:SHOWOUT JOB=@;SP commands). Since :SHOWIN and :SHOWOUT output
can not normally be redirected to a disk file, you should
accomplish this by running a program (say, FCOPY) with its
;STDLIST= redirected and then making the program do the :SHOWIN
and :SHOWOUT, e.g. by saying
:RUN FCOPY.PUB.SYS;INFO=":SHOWOUT JOB=@;SP";STDLIST=...
* Consider the space used by the system directory and by virtual
memory (these values are available using a :SYSDUMP $NULL).
* Consider the space used by your own job's temporary files.
* Run the job when you're the only user on the system.
Your best bet would probably be to run the job once, immediately after
a recover lost disk space, with nobody else on the system; the total
'unaccounted for' disk space it shows you (i.e. the total space minus
the :REPORT sum minus the free disk space) will be, by definition, the
amount of space need by the system and by your job stream. You can
call this post-recover-lost-disk-space value the 'baseline'
unaccounted-for total.
If your job stream ever shows you an 'unaccounted for' total that is
greater than the baseline, you'll know that there MAY be some disk
space lost. To be sure, you should
* Run the job when you're the only user on the system.
* Make sure that your session (the only one on the system) has no
temporary files or open new files while the job is running.
If you do all this, then comparing the 'unaccounted for' disk space
total against the baseline will tell you just how much space is really
lost.
Incidentally, note that you needn't rely on your guess as to the total
size of your disks -- even this can be found out exactly (though not
easily). The very end of the output of VINIT's ">PFSPACE ldev;ADDR"
command shows the total disk size as the "TOTAL VOLUME CAPACITY".
Thus, you could, in your job, :RUN PVINIT.PUB.SYS (the VINIT program
file) with ;STDLIST= redirected to a disk file and issue a ">PFSPACE
ldev;ADDR" command for each of your disk drives. (If you want to get
REALLY general-purpose, you can even avoid relying on the exact
logical device numbers of your disks by doing a :DSTAT ALL into a disk
file and then converting the output of this command into input to
VINIT).
Finally, it's important to remember that all this applies ONLY to
MPE/V. The rules of the game for MPE/XL are very, very different; in
any event, disk space on MPE/XL should (supposedly) never get lost.
Q: I want to "edit" a spool file -- make a few modifications to it
before printing it. SPOOK, of course, doesn't let you do this, so I
decided just to use SPOOK's >COPY command to copy the file to a disk
file, use EDITOR to edit it, and then print the disk file. However,
when I printed the file, the output pagination ended up a lot
different from the way it was in the original spool file! Is there
some special way in which I should print the file, or am I doing
something else wrong?
A: Well, first of all, there is a special way in which you must print
any file that used to have Carriage Control before you edited it with
EDITOR. When EDITOR /TEXTs in a CCTL file, it conveniently forgets
that the file had CCTL -- when it /KEEPs the file back, the file ends
up being a non-CCTL file (although the carriage control codes remain
as data in the first column of the file). To output the file as a CCTL
file, you should say
:FILE LP;DEV=LP
:FCOPY FROM=MYFILE;TO=*LP;CCTL
The ";CCTL" parameter tells FCOPY to interpret the first character of
each record as a carriage control code.
Actually, you probably already know this, since otherwise you would
have asked a slightly different question. You've done the :FCOPY
;CCTL, and you have still ended up with pagination that's different
from what you had to begin with. Why?
Unfortunately, not all the carriage control information from a spool
file gets copied to a disc file with the >COPY command. In particular:
* If your program sends carriage control codes to the printer using
FCONTROL-mode-1s instead of FWRITEs, these carriage control codes
will be lost witha >COPY.
* If the spool file you're copying is a job $STDLIST file, the
"FOPEN" records (which usually cause form-feeds every time a
program is run) will be lost.
* And, most importantly, if your program using "PRE-SPACING"
carriage control rather than the default "POST-SPACING" mode, the
>COPYd spool file will not reflect this.
Statistically speaking, it is this third point that usually bites
people.
The MPE default is that when you write a record with a particular
carriage control code, the data will be output first and then the
carriage control (form-feed, line-feed, no-skip, etc.) will be
performed -- this is called POST-SPACING. However, some languages
(such as COBOL or FORTRAN) tell the system to switch to PRE-SPACING
(i.e. to do the carriage control operation before outputting the data),
and it is precisely this command -- the switch-to-pre-spacing command
-- that is getting lost with a >COPY.
Thus, output that was intended to come out with pre-spacing will
instead be printed (after the >COPY) with post-spacing; needless to
say, the output will end up looking very different from what it was
supposed to look like.
What can you do about this? Well, I recommend that you take the disc
file generated by the >COPY and add one record at the very beginning
that contains only a single letter "A" in its first character. This
"A" is the carriage control code for switch-to-pre-spacing, and it is
the very code that (if I've diagnosed your problem correctly) was
dropped by the >COPY command. By re-inserting this "A" code, you
should be able to return the output to its original, intended format.
Now, this is only a guess -- I'm just suggesting that you try putting
in this "A" line and seeing if the result is any better than what you
had before. There might be other carriage control codes being lost by
the >COPY; there might be, for instance, carriage control transmitted
using FCONTROLs, or your program might even switch back and forth
between pre- and post-spacing mode (which is fairly unlikely).
However, the lost switch-to-pre-spacing is, I believe, the most common
cause of outputs mangled by the SPOOK >COPY command.
Robert M. Green of Robelle Consulting Ltd. once wrote an excellent
paper called "Secrets of Carriage Control" (published, among other
places, in The HP3000 Bible, available from PCI Press at (512)
250-5518); it explains a lot of little-known facts about carriage
control files, and may help you write programs that don't cause
"interesting" behavior like the one you've experienced. Carriage
control is a tricky matter, and Bob Green's paper discusses it very
well.
Q: Please help me resolve a problem I am having attempting to use only
a function key to issue a :HELLO command. I know about the escape
sequence that will define a user function key with the characters for
the :HELLO; the problem I'm having is that the RETURN key must be
entered first before the character string from the function key is
accepted. This might not sound like much of a problem but the
combination of pressing the RETURN key followed by a function key
would be too confusing for the users to follow. What I would like to
do would be to have the users sign on by pressing only a function key.
Security is done by the application, including passwords, so I don't
think that I'm taking that much of a security risk by having the
:HELLO command stored in a function key.
What is apparently happening is that the RETURN key triggers the CPU
to accept a response from the terminal using the DC1, DC2, DC1
protocol; what I can't accomplish is duplicating this pattern as an
escape sequence in the function key definition. The best that I can do
is to include the escape sequence for the RETURN key as the first
character in the string. This will make the CPU accept the remaining
portion of the character string but the time delay causes the
beginning characters of the HELLO command to be missed. I have tried
other escape patterns without any success. Please do what you can to
help me resolve this problem.
A: The ozone layer is dissolving, crime is rampant in the streets, and
the Khmer Rouge are about to regain power in Cambodia -- and THIS you
call a problem? Well, I suppose it is, after a fashion, and being
unable to do anything about the first three problems I mentioned, I
did some research into yours.
Since I know absolutely nothing about this sort of issue (MPE,
languages, and the File System being more my specialty), my research
led me to the people who know EVERYTHING about terminals talking to
the HP3000, namely Telamon, Inc. in Oakland. Randy Medd of Telamon
(one of their two technical gurus, the other being, of course, Ross
Scroggs) was kind enough to answer the question, and here is his
reply:
The only apparent solution to this problem involves, among other
things, the necessity of hitting the function key twice, e.g. [f1]
[f1]. Here is how this can be done:
1. Make sure that the logon port is configured as Speed Specified
(i.e. NOT Speed-Sensing) -- either sub-type 4 (for a
direct-connect) or 5 (for a modem port). This, by the way,
requires that the terminal's configured speed EXACTLY matches
the port's default speed.
2. Define the function key as an 'N' (Normal) type key with an
explicit carriage return terminating the sequence, e.g.
"[ESC]&f1k0a16d019LLog On HELLO USER.ACCOUNT[CR]"
(where [ESC] is an ASCII 27 and [CR] is an ASCII 13). Be sure
that "k0" is specified, not "k1" or "k2".
3. The first press of this function key will cause the ":" prompt
to appear and the second press will cause the HELLO string to
be sent to MPE, although the user must still wait for the ":"
before pressing the function key the second time.
The reason why this won't work on a Speed-Sensed port (sub-types 0
and/or 1) is that the speed-sense algorithm requires some "dead"
time both before and after the carriage return in order to
determine the speed. If the aforementioned function key sequence
were sent to a speed-sensed port, no ":" would be generated as too
many characters arrived too soon BEFORE the first CR was
encountered. If the function key were defined with a leading
carriage return, in addition to the HELLO ... [CR], no ":" would be
generated as the terminal driver would see too many characters
arriving too soon AFTER the initial CR. Note that this synopsis is
based upon observation, not upon inspection of the actual terminal
driver code itself.
If the port is Speed-Specified, the additional characters
encountered during the first transmission of the key will be
ignored as the driver can instantly recognize each character as it
arrives, discarding characters until the first CR is found.
Even the addition of type-ahead (via a Type Ahead Engine,
Reflection, or Advance Link) won't solve this problem. All three of
these products initially assume that their type-ahead function will
not be activated until the initial DC1 read trigger is encountered
from the HP3000. Programming a function key with a leading CR will
fail on a Speed-Specified port because the entire function key's
contents will almost certainly be transmitted to the HP3000 before
the computer gets around to sending back the initial CR+LF+":"+DC1
prompt for the logon (although the [F1] [F1] sequence should work).
This sequence will fail on a Speed-Sensed port for the same reason
mentioned above -- no ":"+DC1 will be generated as the speed-sense
algorithm won't recognize any CRs sent with text immediately
preceding or following.
So, there it is, thanks to Randy. It seems to me that you might be
better off convincing your users that they can, after all, master
hitting the RETURN key before using the function key -- otherwise,
you'll have to reconfigure your ports to be Speed-Specified, which
will be at least a bother and perhaps a serious problem, since you
would then have to reconfigure the system any time you want to change
a terminal baud rate (which, to be frank, should be rather rare).
If you MUST avoid that leading RETURN key, it seems that Randy's
double-hit-of-the-function-key solution is your best bet. Good luck.
Q: When we got V-delta-4, we started to use the Access Control
Definition (ACD) feature, which finally let us restrict access to
files to specific users. We were very happy with it, and in fact set
up ACDs for thousands of our key files. Everything went well until our
next reload -- after the reload was done, we had found that all the
ACDs had VANISHED!!! What happened???
A: If you take a careful look at the V-delta-4 Communicator, you will
find (among the twenty or so pages devoted to ACDs) the following
paragraph (on the bottom of page 3-2):
If the MPE V/E command :SYSDUMP is used to perform a backup, the
;COPYACD parameter must be specified in the dump file subset:
:FILE T;DEV=TAPE
:SYSDUMP
Any changes?
Enter dump date? 0
Enter dump file subset
@.@.@;COPYACD
Also, at the bottom of page 3-16, HP says:
To backup files with ACDs in :SYSDUMP, the ;COPYACD keyword will
need to be specified with the fileset information.
If you do NOT specify ;COPYACD on a :SYSDUMP (or on a :STORE), THE
ACD'S OF THE STORED FILES WILL *NOT* BE WRITTEN TO TAPE. Then, if you
:RESTORE or RELOAD from the tape, the ACDs will be COMPLETELY LOST
(since there'll be no place for the system to find them).
So, there it is -- as Alfredo Rego once put it, documentation should
be read like a love letter, with attention paid to every dot over
every 'i' and every crossing of a 't'. If you miss those couple of
paragraphs, you lose all your ACD information the next time you reload
or :RESTORE from a backup.
You may well ask: If ;COPYACD is so vital for all ACD users, WHY LEAVE
IT AS AN OPTION??? Well, HP had a reason for it: compatibility. Tapes
with ACDs on them are NOT readable by pre-V-delta-4 versions of
:RESTORE (or of the RELOAD option of the system boot facility). If HP
had defaulted all post-V-delta-4 :STOREs and :SYSDUMPs to have the
;COPYACD option, then you might have run into problems the next time
you tried to transport a post-V-delta-4 tape to a pre-V-delta-4
machine. I'm not sure that this would be a more severe problem than
the total loss of all ACDs that may happen with the default being what
it is now, but I can certainly see HP's point.
So, if you use ACDs, you MUST put a ;COPYACD on your system backup
command, but realize at the same time that this will make your tapes
unreadable on pre-V-delta-4 systems.
The :FULLBACKUP and :PARTBACKUP commands, incidentally, will always
create a backup tape with the ;COPYACD option. This will avoid
confusion with lost ACDs, but, as we just pointed out, will make their
backup tapes untransportable to pre-V-delta-4 machines. You can't have
everything.
Q: I'm sick and tired of having to manually look up all
those CI and file system error numbers that my programs print out
whenever they get a COMMAND intrinsic or file system intrinsic error.
How can I make them print the error MESSAGE, not just the error
NUMBER?
A: I agree wholeheartedly; I got tired of manually looking up errors
years ago (especially since it's hardly easy to look them up).
Fortunately, there is a -- relatively -- straightforward way of
turning those cryptic numbers into (sometimes equally cryptic) error
messages.
On MPE/V, the intrinsic that you want is called GENMESSAGE. Its job is
to output an arbitrary message from an arbitrary "set" of an arbitrary
message catalog file. Actually, you can use this to create your own
catalog files with your own programs' error messages, but for your
particular problem you need to use the file CATALOG.PUB.SYS.
CATALOG.PUB.SYS is where MPE keeps virtually all the messages that it
can output to you -- console messages, CI error messages, file system
error messages, loader error messages, etc. (You can even modify the
text of these messages by modifying the CATALOG.PUB.SYS file -- see
Chapter 9 of the MPE V System Operation and Resource Management
Manual.) To print a particular message from this file, you must open
this file MR NOBUF and then call the GENMESSAGE intrinsic, passing the
file number, the error number whose message you want to print, and the
"set number", which is 2 for CI errors and 8 for file system errors.
Thus, here's what a sample error message printing procedure might look
like (in this example, it's in FORTRAN, but it could be in any
language):
SUBROUTINE PRINTMPEERROR (FNUM, ERRNUM)
INTEGER FNUM, ERRNUM
SYSTEM INTRINSIC FOPEN, GENMESSAGE
IF (FNUM .NE. 0) GOTO 10
FNUM = FOPEN ("CATALOG.PUB.SYS ", 1, %420)
10 CALL GENMESSAGE (FNUM, 2, ERRNUM)
RETURN
END
If you pass it a file number of 0, it automatically opens
CATALOG.PUB.SYS and returns its file number; subsequent calls won't
require CATALOG.PUB.SYS to be re-opened. A similar procedure could be
written to print a file system error message; it would just have to
pass an "8" instead of a "2" to GENMESSAGE.
2 and 8 are the arbitrary set numbers that the designers of
CATALOG.PUB.SYS chose for MPE errors and file system errors,
respectively; they're the most useful of the more than 20 sets in the
file, but you can find out about the other message sets by looking at
all the lines in CATALOG.PUB.SYS that start with "$SET ".
What about MPE/XL? The GENMESSAGE intrinsic is still the right way to
output CI errors and FOPEN errors on MPE/XL; however, many intrinsics
(such as HPFOPEN, HPCIGETVAR, and others) return a 32-bit "status
word" that GENMESSAGE can't handle. To output this sort of status
word, you should call the new HPERRMSG intrinsic, e.g.:
PROCEDURE HPERRMSG; INTRINSIC;
...
HPERRMSG (2, 0, 0, STATUS);
(This example is in PASCAL/XL, but HPERRMSG is callable from any
language.) The "2" simply indicates that the error message is to be
printed to the terminal; there's no need to specify a set number,
since the MPE/XL 32-bit status contains within itself an indication of
what kind of error this is.
What if you've been given an error number -- a CI error, a file system
error, or an MPE/XL 32-bit status -- and want to find out the
corresponding error message without writing a program? There are some
contributed programs that do this; you can also (for CI and file
system errors) say:
:FCOPY FROM=CATALOG.PUB.SYS;TO;SUBSET="1234 "
This will show you all the lines in CATALOG.PUB.SYS that start with
(in this example) "1234 ", which will include all the error messages
(CI, file system, loader, etc.) corresponding to error #1234; a clumsy
approach, but better than nothing. (Of course, you can also use your
favorite editor for this.)
If you use MPEX/3000, you can say
%CALC ABORTMPEF (cierr, fserr)
which will output the error messages corresponding to the given CI
error number and file system error number; if you only want to output
one of these errors, you can set the other parameter to 0, e.g.
%CALC ABORTMPEF (0, 74)
to get the message for file system error 74. (Don't ask me why I
called this function ABORTMPEF instead of something sensible.)
An upcoming version of MPEX/3000 will also have a WRITEMPEXLSTATUS
function that will let you say (on MPE/XL)
%CALC WRITEMPEXLSTATUS (statusvalue)
to format an MPE/XL status.
Q: Suppose I want an application to access the :RUN ...;INFO=
information. I would like to use the GETINFO system intrinsic, but
only if it is available (since some early-MPE/V and MPE/IV machines
don't have it). If GETINFO isn't available, I don't mind not having
access to the ;INFO= string (and the ;PARM= value), but I do want the
rest of the program to run; however, if I just call GETINFO and it
isn't there, my program won't even load.
I suspect that what I want can be accomplished with the LOADPROC and
UNLOADPROC system intrinsics, but I don't know exactly how.
A: You're right -- LOADPROC is the very thing that you need, in this
case, or in any other case in which you want to call a procedure that
may or may not be there (for example, a new MPE/XL procedure [such as
the Switch procedure, HPSWTONMNAME], if you have both an MPE/V and an
MPE/XL machine).
Calling the LOADPROC intrinsic is, however, the easy part. You pass to
it the procedure name (a byte array, terminated with a blank) and a
library identifier (0 indicates to load from SL.PUB.SYS) -- it returns
to you an "identity number" (which you can use when you call
UNLOADPROC) and the procedure's "plabel". This much the Intrinsics
Manual will tell you; what it won't tell you is HOW TO CALL THE
PROCEDURE once you've loaded it!
That's right -- LOADPROC doesn't actually call the procedure (how can
it if you haven't passed the procedure parameters?), but just returns
the procedure's "plabel". The trick now -- and it's a very tricky
trick -- is to use this plabel to call the procedure. This is
something that you have to do in SPL, and is one of the rare times in
which you MUST use the low-level ASSEMBLE and TOS constructs that the
SPL compiler gives you.
To call a procedure given its plabel, you must set up all of the
procedure's parameters on the stack in EXACTLY the way that a compiler
would if you called the procedure normally from the compiler; then,
you must push the plabel onto the stack (say TOS:=PLABEL) and call the
procedure (using ASSEMBLE (PCAL 0)).
This is best shown by an example:
INTEGER PROCEDURE MYGETINFO (INFO, INFOLEN, PARM);
BYTE ARRAY INFO;
INTEGER INFOLEN, PARM;
<< Returns 0 if all went well, >>
<< 1 if GETINFO failed or couldn't be loaded. >>
<< To call from COBOL, be sure to specify an "@" before >>
<< the INFO parameter, and declare INFOLEN and PARM >>
<< as PIC S9(4) COMP. >>
BEGIN
INTRINSIC LOADPROC, UNLOADPROC;
BYTE ARRAY PROC'NAME(0:15);
INTEGER IDENT, PLABEL;
MOVE PROC'NAME:="GETINFO ";
IDENT:=LOADPROC (PROC'NAME, 0, PLABEL);
IF <> THEN
MYGETINFO:=1
ELSE
BEGIN
TOS:=@INFO;
TOS:=@INFOLEN;
TOS:=@PARM;
TOS:=%7; << guess what this means >>
TOS:=PLABEL;
ASSEMBLE (PCAL 0);
MYGETINFO:=TOS;
UNLOADPROC (IDENT);
END;
END;
The LOADPROC call is simple; if it fails (condition code <>), we
return immediately with a result of "1". What if it succeeds?
* We push the three GETINFO intrinsic parameters onto the stack.
Note that we say "TOS:=@varname" -- this means to push the
ADDRESS of the given parameter onto the stack; we must do this
because all three of the GETINFO intrinsic's parameters are by
REFERENCE. If they were by VALUE, we'd just push them onto the
stack by saying "TOS:=varname" or "TOS:=expression".
* We push a 7 onto the stack; the 7 is the GETINFO intrinsic's
OPTION VARIABLE MASK. The GETINFO intrinsic is marked as "O-V" in
the Intrinsics Manual, which means that some of its parameters
may be omitted. Even though we don't omit any in this call, the
intrinsic does expect -- at the very top of the stack -- the bit
mask that indicates which parameters are actually being passed.
The lowest-order bit (bit 15) indicates the presence (1) or
absence (0) of the LAST parameter; the next lowest (14)
corresponds to the SECOND TO LAST; and so on. The 7 (in which
bits 13, 14, and 15 are set) simply means that all 3 parameters
are passed.
The few procedures that take more than 16 parameters expect a
TWO-word (32-bit) OPTION VARIABLE mask; procedures with 16 or
fewer parameters expect just a one-word mask.
* We push the PLABEL onto the stack and then do an ASSEMBLE (PCAL
0) -- the "PCAL 0" means "call the procedure whose plabel is on
top of the stack". This actually calls the GETINFO intrinsic.
* Finally, since GETINFO returns a result, the result remains on
the stack after the PCAL 0; the MYGETINFO:=TOS pops this value
into the MYGETINFO procedure result.
So, there you go; all we need to do now is an UNLOADPROC and we're
done. Of course, as I mentioned, this can only be done in SPL (unless
you know of a TOS or ASSEMBLE operator in COBOL!); and, each procedure
you want to call this way requires some special thinking, since it's
very easy to make a mistake in the way you push the parameters onto
the stack.
However, once you master this, you can write programs that can take
advantage of powerful features of newer versions of MPE while still
remaining compatible with older versions. Even if all of your
computers have the GETINFO intrinsic (the great majority of HP3000s
now do), this can still be relevant for any new intrinsics that HP
comes out with, for instance HPACDPUT or HPACDINFO on MPE/V, or
HPSWTONMNAME on MPE/XL.
This mechanism is also quite useful when you want to call a procedure
whose name is not known at the time you write your program; for
example, EDITOR's and QEDIT's /PROCEDURE commands let the user specify
the name of his own procedure that can be called for sophisticated
editing operations. One thing that you have to remember for this,
though, is that you have to firmly dictate to the user the exact
calling sequence of the procedure that he is to write, since you'll
have to properly stack all the parameters for the procedure call.
Finally, note that on MPE/XL in Native Mode the mechanism for doing
this is substantially different, and substantially simpler;
PASCAL/XL's CALL and FCALL built-in functions and the UNRESOLVED
procedure attribute let you do all this without any ASSEMBLE or TOS
mumbo-jumbo.