MPE PROGRAMMING
by Eugene Volokh, VESOFT
Presented at 1983 HPIUG Conference, Montreal, PQ, CANADA
Published by HPIUG Journal, Apr/June 1983 (Vol.6, No.2).
Published in "Thoughts & Discourses on HP3000 Software", 1st-4th ed.
ABSTRACT
MPE, with its limited control structures and small set of commands,
may not at first seem to be a powerful system programming tool.
However, it turns out that MPE can be as powerful as (and easier to
use than) any programming language for certain system programming
tasks. This paper will try to introduce the reader, via a series of
examples, to the art of "MPE programming."
THREE EXAMPLES OF MPE PROGRAMMING IN ACTION
Recently, in my capacity as systems consultant to a large HP
installation, I encountered the following situation:
There is a large system that can operate in one of two modes --
ONLINE or BATCH. The mode in which it operates is indicated by the
presence or absence of a certain file called HOL100. If this file
exists, this means that the system is operating in an ONLINE mode;
if it does not, the system is operating in a BATCH mode.
It is desirable to print at logon time the mode in which the system
is running.
Obviously, since something is to be done at logon time, we should use
a logon UDC. The simplest solution is to have one of the form:
LOGONUDC
OPTION LOGON
RUN CHKSTAT
where CHKSTAT is a program that checks whether HOL100 exists and
prints out an appropriate message. However, this is not the best
solution. For one, I don't feel like writing a custom SPL program
every time a rather simple systems programming task comes around;
those who are not familiar with FOPEN will find this even harder to
do. Furthermore, even if I did write a custom program for this,
either the program or the source file is virtually guaranteed to get
lost. And finally, running a program is a rather long and
resource-consuming task.
But, if not a program, then what? After all, MPE does not even have a
DISPLAY command to print a message, much less a command that will
check whether a file exists and display one message if it does and
another if it doesn't.
At this point, I must make a confession: despite what I said of the
possibilities of MPE as a systems programming language, it was by no
means created to be a systems programming language. In fact, you will
find that most of the techniques that will be described are actually
methods of subverting MPE commands to do tasks that they were never
intended to do in the first place. However, they work, and that's
what counts.
Returning to the problem at hand, let us attack it one step at a time.
For one, as I said, MPE does not provide us with a DISPLAY command.
So, we'll make one!
UDCs are permitted to have a number of options. One of these options,
LIST, instructs MPE to list out the commands in the UDC as they are
executed. Furthermore, there is an MPE command called :COMMENT that
does absolutely nothing. So, what do we get when we cross an OPTION
LIST and a command that does nothing?
DISPLAY !STRING
OPTION LIST
COMMENT !STRING
When the above UDC is invoked via a command of the form 'DISPLAY
"string"', it will execute the command 'COMMENT string' (which in and
of itself will do nothing), but also list this command as it is being
executed! Thus, if we don't mind seeing 'COMMENT' on the screen, we
now have a way of displaying anything we want to on the terminal from
within a UDC.
We've licked one of our problems -- we can now display a message to
the terminal. However, this still does not solve the other problem --
determining whether a file exists or not and printing one message if
it does and another if it doesn't.
Here, we must introduce a very important MPE construct (in fact, its
only control structure) -- the :IF command. With the :IF command and
its two sidekicks, :ELSE and :ENDIF, we can, depending on the value of
a logical expression, execute one of two sets of commands. Thus, our
task can be expressed as follows:
LOGONUDC
OPTION LOGON
Check if HOL100 exists
IF it exists THEN
DISPLAY "USING THE ONLINE SYSTEM"
ELSE
DISPLAY "USING THE BATCH SYSTEM"
ENDIF
However, even before you start to furiously leaf through your MPE
commands manual, you will probably begin to suspect that neither
'Check if HOL100 exists' nor 'it exists' is valid MPE syntax. In
fact, there is no check-if-a-file-exists command in MPE. Or is there?
Well, if there is no command that will explicitly check whether a file
exists, we ought to look for a command that, as a side effect, yields
different results depending on whether a file exists or not.
Furthermore, we would be able to differentiate these results using an
:IF command.
Let us consider the :LISTF command. If we do a ':LISTF filename', the
filename will be listed if the file exists, and a CI error 907 will be
generated if it does not. Since we want as little output to the
terminal as possible, we actually want to do a ':LISTF filename;
$NULL', which will do nothing if the file exists, and print a CI error
907 if it does not. Furthermore, it turns out that the value of the
last CI error is stored in a JCW (Job Control Word) called CIERROR,
which can be interrogated via the :IF command. Thus, instead of
'Check if HOL100 exists' we should say ':LISTF HOL100; $NULL' and
instead of 'it exists' we should say 'CIERROR<>907.' Thus, the
solution to our problem is:
LOGONUDC
OPTION LOGON
SETJCW CIERROR=0
CONTINUE
LISTF HOL100;$NULL
IF CIERROR<>907 THEN
DISPLAY "USING THE ONLINE SYSTEM"
ELSE
DISPLAY "USING THE BATCH SYSTEM"
ENDIF
A few comments: 'SETJCW CIERROR=0' makes sure that CIERROR is cleared
before the :LISTF command. Since this is an OPTION LOGON UDC, it is
guaranteed to be zero anyway, but in general it is conceivable that it
was already 907 before the :LISTF command. More importantly, a
:CONTINUE command was added before the :LISTF command to avoid the UDC
aborting on the first error; a :CONTINUE (either in a UDC or a job
stream) instructs MPE not to abort if the next command fails.
One other point: in addition to the appropriate message, this method
leaves some junk on the screen, namely the LISTF command and the error
message if the file does not exist (and thus the LISTF command failed)
and in either case a 'COMMENT' from the DISPLAY UDC. This is actually
rather easy to take care of -- merely embed in the DISPLAY string some
escape sequences to move the cursor and delete the unwanted lines and
characters on the screen. If you're using printing terminals, though,
you're out of luck.
Thus, we have seen how using MPE alone we can perform some fairly
complex tasks easily and efficiently.
So, from this, we can derive a sort of MPE programming methodology:
1. If you see no direct way of performing a given task, try to find a
way that yields the desired effect as a side effect, with little
or no other direct effects or side effects.
2. If you wish to do two different things depending on some condition
that cannot be straightforwardly expressed with JCWs, try to find
a command or sequence of commands that yields two different JCW
values depending on the condition.
Let us take another example:
One of VESOFT's products, MPEX, is an extended MPE user interface
that provides many desirable features, and is often 'lived in' by
its users -- they run it once when they sign on and stay in it until
they are done, at which time they exit it and immediately sign off.
Some of our users decided to set up an option logon UDC of the form
MPEX
OPTION LOGON
RUN MPEX.PUB.VESOFT
BYE
This way, they would be automatically dropped into MPEX when they
sign on, and would automatically be :BYEd off when they exit it.
However, they do not want this to be done for jobs, but rather only
for sessions. Thus, the task is to determine within a UDC whether
the user is in a job or a session.
In my opinion, in addition to the already existing system-defined JCWs
such as JCW and CIERROR, HP should have provided us with JCWs such as
MODE (to indicate whether we are a session or a job), FSERROR, etc.
However, the fact remains that it did not, and we have to determine
this for ourselves.
Let us apply our rule #2 -- is there a command that yields somewhat
different results for job mode and session mode? In fact, there is.
The :RESUME command, when executed from within session mode (but not
from break mode, since the UDC will never be executed from within
break mode) yields a CIWARN 1686 (COMMAND ONLY ALLOWED IN BREAK);
however, when executed from within job mode, it issues a CIERR 978
(COMMAND NOT ALLOWED IN JOB MODE). Furthermore, since this is an
OPTION LOGON UDC and will therefore never be executed from break mode,
the RESUME command has no other effects! Thus our solution would be:
MPEX
OPTION LOGON
SETJCW CIERROR=0
CONTINUE
RESUME
IF CIERROR<>978 THEN
RUN MPEX.PUB.VESOFT
BYE
ENDIF
As an additional nicety, we may wish to do something like a 'DISPLAY
"PLEASE IGNORE THE FOLLOWING MESSAGE"' before the RESUME command so
that the user will not be puzzled by the warning that the RESUME
command issues in session mode.
So, score another point for UDC programming.
To round out this section, consider one more example:
Before performing a given task, we wish to find out whether a given
file is in use or not. If it is not in use, we should perform the
task; if it is in use, we should print an error message.
Solving this problem requires a substantial amount of knowledge of the
file system. What we really want to do is to try to open the file
with EXCLUSIVE, INPUT access; if the open succeeds, we want to close
the file with SAVE disposition; if it fails, we want to set a flag.
However, we cannot explicitly open and close files in MPE. Rather, we
have to find a command to subvert so that it would do this task for
us. This command's operation should be essentially similar to our
target operation (i.e. it should do an open followed by a close). One
command that pops to mind is the :PURGE command. Unfortunately, it
opens a file with OUT access and closes it with DEL disposition.
But, via the :FILE command, we can force it to open and close the file
with whatever options we please! Thus, our task may be achieved by
doing the following:
FILE F=filename;EXC;ACC=IN;SAVE
SETJCW CIERROR=0
CONTINUE
PURGE *F
IF CIERROR=384 THEN
DISPLAY "ERROR: FILE IS IN USE"
ELSE
the file is not in use; do whatever is necessary
ENDIF
RESET F
Note that any open failure (except 'nonexistent file') during a :PURGE
command causes a CIERR 384; furthermore, the last file system error is
not accessible as a JCW, so we have to assume that no other open
failure will occur.
ADVANCED MPE PROGRAMMING
Consider the following problem:
VESOFT distributes its products on a tape along with an installation
job stream. When a user wishes to install the products, he
:RESTOREs the job stream and streams it. The job stream creates the
appropriate accounting structure, and then :RESTOREs all the
relevant files off the installation tape. However, it is possible
that some files cannot be restored; in this case, we want to send an
appropriate message to the console.
The obvious thing to do here would be to check CIERROR to see if
:RESTORE failed, and if so, do a :TELLOP. But, :RESTORE does NOT set
CIERROR if all files were not restored! It merely prints the
filenames and the count of the files that were not restored to its
list file, and terminates just like all files were restored.
We have run into a problem that we can't really solve with the
techniques outlined above because no MPE command can examine the
contents of a file for us. However, there is one HP utility that is
made explicitly for examining the contents of files -- EDITOR!
Our plan of attack will be as follows: we will redirect the listing of
the :STORE command to a disc file (by setting a file equation for
SYSLIST), massage it with EDITOR, somehow cause EDITOR to set a JCW
depending on the number of files not stored, and then, when we're back
in MPE, examine that JCW.
So, our "program" will look like this:
:FILE SYSLIST,NEW;DEV=DISC;REC=-80,16,F,ASCII;NOCCTL;TEMP
:RESTORE *VESOFT; @.@.VESOFT, @.@.SECURITY; SHOW; OLDDATE
:RESET SYSLIST
:SETJCW FILESNOTRESTORED=0
:EDITOR
TEXT SYSLIST
LIST ALL
CHANGE "FILES NOT RESTORED",":SETJCW FILESNOTRESTORED",ALL
DELETEQ 1/*-1,*+1/LAST
KEEP $NEWPASS,UNNUMBERED
USE $OLDPASS
EXIT
:IF FILESNOTRESTORED<>0 THEN
: TELLOP SOME FILES NOT RESTORED, CHECK SPOOL FILE!
:ENDIF
What does this mess do? Well, the first three lines do a :RESTORE,
redirecting the listing to a disc file. Then, we enter EDITOR and
text in the list file. Now, we have to make EDITOR set a JCW
depending on the number of files not restored. The way that we do
this is by changing the 'FILES NOT RESTORED = xxx' line to ':SETJCW
FILESNOTRESTORED = xxx' with the CHANGE statement, deleting all the
other lines in the file, keeping this as a temporary file, and USEing
this file! The USE command will read the file and execute the :SETJCW
command that we put in it; now, when we exit EDITOR, the
FILESNOTRESTORED JCW is equal to the number of files not restored, and
can now be interrogated.
This kind of trick is a very valuable one, and should be added to our
methodology:
3. If the parameters of an MPE command (in this case :SETJCW) depend
upon the result of another MPE command (in this case :STORE),
redirect the listing of the latter into a disc file, and use
EDITOR to create and execute the former. Similarly, if the input
of a program depends on the result of another program or command,
redirect the listing of the latter into a disc file, and use
EDITOR to create the input file for the former.
This point is best explained by another example:
VINIT, an HP utility, has a '>PDTRACK LDEV' command, which prints
the addresses of all the defective disc tracks on the disc device
indicated by LDEV. However, VINIT has no '>PDTRACK ALL' command.
Implement it.
Applying our methodology, our strategy should be:
A. Find a command that lists all the disc devices in the system, and
redirect its output to a disc file.
B. Using :EDITOR convert this output into input for VINIT.
C. Run VINIT using this newly-generated input file.
For step A, one command that seems to fit the bill is :DSTAT ALL. This
little-known command produces output of the form:
LDEV-TYPE STATUS VOLUME (VOLUME SET-GEN)
----------- --------- -------------------------
1-7925 SYSTEM MH7925U0
2-7925 SYSTEM MH7925U1
3-7925 SYSTEM MH7925U2
As you see, this command displays, among other things, the logical
device numbers of all the discs in the system. However, one problem
comes up immediately: unlike the :STORE command, whose output can
easily be redirected to a disc file, :DSTAT ALL's output always goes
to $STDLIST.
So, how are we going to redirect the output of a command that can only
send its output to $STDLIST? The answer is simple: redirect $STDLIST!
Although we cannot redirect the $STDLIST of a job or of a command, we
can redirect the $STDLIST of a program. So, all we need to do is to
issue the following commands:
:FILE LISTFILE,NEW; REC=-80,,F,ASCII; NOCCTL; TEMP
:RUN FCOPY.PUB.SYS; STDLIST=*LISTFILE
:DSTAT ALL
EXIT
:RESET LISTFILE
What we do is run FCOPY with its $STDLIST redirected to a disc file,
and cause it to do a :DSTAT ALL. :DSTAT ALL will obediently print its
output to $STDLIST, which has been redirected!
So, we have the :DSTAT ALL listing (along with some other stuff
printed by FCOPY) in a temporary disc file called LISTFILE. Now, it is
time for step B -- converting this :DSTAT ALL list file to a VINIT
input file:
:FILE INFILE;TEMP
:EDITOR
TEXT LISTFILE
DELETE 1/6,LAST << delete the various headers >>
FIND FIRST
WHILE
FIND "-" <<delete all after the '-',>>
DELETE *(*)/*(LAST) <<leaving only the LDEV>>
CHANGE 1,"PDTRACK",ALL <<insert PDTRACKs before LDEVs>>
ADD <<add an EXIT command>>
EXIT
//
KEEP *INFILE
EXIT
We now have the VINIT input file; all we need to do is
:FILE INFILE,OLDTEMP
:RUN PVINIT.PUB.SYS;STDIN=*INFILE
and we're done!
We finish off this section with one more example:
VESOFT's installation stream signs on as MANAGER.SYS, builds the
VESOFT and SECURITY accounts and streams two jobs, which sign on as
MANAGER.VESOFT and MANAGER.SECURITY and build the VESOFT and
SECURITY accounts. It is also the duty of the MANAGER.SYS job
stream to restore the VESOFT and SECURITY files; however, it can not
do this until the other two jobs finish. How can we make the
MANAGER.SYS job stream wait for the others to terminate?
The key word in this problem is 'wait.' Again, on the surface it
seems that MPE has no command that permits one to wait for a certain
event to occur. Again, however, a trick exists that saves the day.
This trick uses MESSAGE FILES.
Message files are a kind of file (introduced in MPE IV) that have the
property that if a reader tries to read an empty message file, he does
not get an immediate end of file, but rather suspends until the
message file is no longer empty.
So, even before the two job streams are streamed, we build two message
files in PUB.SYS: MSGVESOF and MSGSECUR. Furthermore, in each of the
two internally streamed job streams, we write a record (via FCOPY) to
the appropriate message file after we are all done. And, in the main
(MANAGER.SYS) job stream, right after we stream the two other job
streams but before we do the :RESTORE, we read the two message files
(again, via FCOPY). The resultant job stream goes like this:
!JOB MANAGER.SYS
!NEWACCT VESOFT
!NEWACCT SECURITY
!BUILD MSGVESOF
!RELEASE MSGVESOF <<so the job stream can write to it>>
!BUILD MSGSECUR
!RELEASE MSGSECUR
!STREAM ,# <<stream the two other job streams>>
#JOB MANAGER.VESOFT
...
#FCOPY FROM;TO=MSGVESOF.PUB.SYS
VESOFT ACCOUNTING STRUCTURE BUILT! <<any message will do>>
#EOJ
#JOB MANAGER.SECURITY
...
#FCOPY FROM;TO=MSGSECUR.PUB.SYS
SECURITY ACCOUNTING STRUCTURE BUILT!
#EOJ
!FCOPY FROM=MSGVESOF;TO <<wait for the VESOFT stream>>
!FCOPY FROM=MSGSECUR;TO <<wait for the SECURITY stream>>
!RESTORE ...
!EOJ
The message file reads cause the job stream to suspend until the
message files are no longer empty, i.e. until the other job streams
have written something to them. Thus, when the :RESTORE is executed,
we are assured that the VESOFT and SECURITY accounting structures have
been built.
CONCLUSION
I have presented some examples and some guidelines that should give
the reader an idea of what MPE programming can do and how it can do
it. It is my belief that with this knowledge and some ingenuity, the
reader can use the art of MPE programming to his advantage.
As another example of MPE programming see our stream LISTAUG:
:JOB LISTAUG,MANAGER.SYS; OUTCLASS=,1; HIPRI
:COMMENT **************************************************
:COMMENT: --LIST OF USERS & GROUPS WITHIN EACH ACCOUNT--
:COMMENT: by Eugene Volokh
:COMMENT: VESOFT, Inc.
:COMMENT: 1135 S. Beverly Dr.
:COMMENT: Los Angeles, CA, 90035-1119 USA
:COMMENT: tel 310-282-0420; fax 310-785-9566
:COMMENT **************************************************
:
:TELLOP DO A :HEADOFF 6, PLEASE...
:FILE LISTAUG;DEV=LP,11,1
:FILE NEWPASS=$NEWPASS;NOCCTL;REC=-80,16,F,ASCII
:REPORT NOGROUP.@,*NEWPASS
:EDITOR
SET QUIET
TEXT $OLDPASS
DELETEQ 1/2
CHANGE 14/80 TO "" IN ALL
FINDQ FIRST
WHILE FLAG
BEGIN
CHANGE 1 TO "LISTACCT "
HOLD *
ADD *,HOLD,NOW
CHANGE "LISTACCT " TO "LISTGROUP @."
ADD *,HOLD,NOW
CHANGE "LISTACCT " TO "LISTUSER @."
FIND *+1
END
CHANGE 40 TO ",*LISTAUG" IN ALL
ADD
EXIT
//
:FILE STDIN;TEMP
KEEP *STDIN
EXIT
:REPORT NOGROUP.@,*LISTAUG
:FILE STDIN,OLDTEMP
:RUN LISTDIR2.PUB.SYS;STDIN=*STDIN
:TELLOP DO A :HEADON 6, PLEASE...
:EOJ