THE SECRETS OF SYSTEM TABLES... REVEALED!
by Eugene Volokh, VESOFT
Presented at 1985 HPIUG Conference, Washington, DC, USA
Published by SUPERGROUP Magazine, Nov 1984/Apr 1985.
Published in "Thoughts & Discourses on HP3000 Software", 3rd ed.
The System Table is an Ornery Beast
That defies investigation
Unless one first acquaints oneself
With the proper incantations.
Long must one labor o'er
Deep-hidden books of arcane lore
Until one learns the secrets of
EXCHANGEDB, MFDS, and more.
And formats one must understand
What's in word one, what's in word two
What lurks inside the fourteenth bit
And what the sixteenth SIR can do.
A systems programmer's what Wilfred was,
With stolid heart and trusty blade,
And system tables he did read,
And many useful programs made.
Oh, listen to his fearsome tale
Of magics dark and dragons fierce,
And catch a glimpse of mysteries
Not often told upon this earth.
-- "The Saga of Wilfred,"
Norse, ca. 11th century AD.
ABSTRACT
MPE's System Tables contain a large variety of data that can be very
useful for any task from performance optimization to writing system
management utilities to improving system security and adding power to
application programs. Unfortunately, nowhere is it clearly explained
(or for that matter, explained any way at all) how one can access all
these system tables. The goal of this paper is to shown how one can
access data segments, file labels, absolute memory locations,
disc-resident system tables -- in general, all forms of system tables
-- so that armed with the information gleaned from here and a System
Tables Manual, a programmer can sit down and start writing programs
that manipulate data segments.
INTRODUCTION
Just like any other program, the operating system must keep track of
its current state; information on all the things known to it -- files,
jobs, programs, everything -- must be kept somewhere. A "System Table"
is just a name for a data structure containing such data. System
tables may be stored on disc, in memory as a fixed array of memory
locations (e.g. locations %1000 to %1377 of bank 0), in memory as a
data segment or a portion thereof, in the stack of some system process
or even a user process -- any data structure maintained by the
operating system can be legitimately called a system table.
There are two problems with using system tables, which account for the
fact that few people know how to use them safely -- one is that the
format of the tables is little-known, and is usually documented only
in the System Tables Manual (not the most readable document known to
man) or, worse, only in the source code of the procedures that
actually maintain the data structure. Even when the format is
documented (e.g. "word 7 contains the flumbrug framastat" is a typical
example), various important details (just what is a flumbrug
framastat) are omitted. Equally bad is the other major problem -- even
if one knows the format of a system table, nowhere does it clearly
describe how to access it.
This paper will try to address both these problems. It will give
instructions on how to access various system tables, but will also
explain what the various system tables are and how they are
inter-related. This, coupled with the precise formats given in the
System Tables Manual, should allow the interested reader to learn to
manipulate much useful system information. If you don't have the
System Tables Manual, you can order it from HP as part number
32002-90003 (MPE IV) or 32033-90010 (MPE V).
Note that although all the examples in this paper are given in SPL,
there is no reason why you can't write system table-accessing programs
in PASCAL, FORTRAN, or even COBOL. All that is necessary is to write
simple SPL routines for performing those operations (such as MFDS or
TURNOFFTRAPS) that can only be done in SPL, and then put the routines
in an RL or an SL. That way, the routines will be callable from your
favorite language. System procedures like FLABIO, ATTACHIO, or GETSIR
can already be called from other languages.
"WAITER, THERE'S A SYSTEM TABLE IN MY STACK!"
It might come as some surprise to you that one place the system hides
a system table is in your very own stack. Your stack is a "data
segment," a contiguous chunk of memory no more than 32K words long, in
which addressing is done relative to the "data base" (DB) register
(not to be confused with IMAGE databases). If you draw your stack
(with addresses growing toward the top), it would look something like
this (see Section I of SPL Manual):
High memory
Z > --------------------------------------------- ^
| unused space | ^
S > --------------------------------------------- ^
| operands of machine instructions | ^
| and local variables of the currently | ^
| executing procedure | ^
Q > --------------------------------------------- ^
| local variables of other procedures | ^
| and stack markers indicating calls from | ^
| one procedure to another | ^
Qi > --------------------------------------------- positive
| global variables | addresses
DB > ---------------------------------------------
| "DB negative area," accessible only by | negative
| SPL procedures (like V/3000) -- usable | addresses
| for global storage | v
DL > --------------------------------------------- v
| The Nether Regions, where mortals may | v
| not stray and non-privileged accessors | v
| are punished with bounds violations | v
--------------------------------------------- v
Low memory
At the very top is the Z register (stands for Zounds?), which is the
uppermost boundary of the stack data segment (note that in some
manuals, HP draws the stack upside down, with DL on top and Z on
bottom -- keep this in mind when comparing this picture with others).
Then, below it is the S register, which marks the "top of the stack";
parameters to machine instructons are kept at and around S. Below S is
the Q register, which indicates the top of the stack at the time the
currently-executing procedure was called; the procedure's local
variables are allocated above it, with the procedure's parameters and
the stack marker (indicating the place from which the procedure was
called) immediately below it.
"Qi" is actually not a register, but rather the initial value of the Q
register (which changes with every procedure call and every procedure
exit). DB is the base register, from which all addressing is done.
Word address 6680 is actually "DB+6680"; any pointers that are kept
(even if they are pointers to local variables, which are between the Q
and S registers) are relative to DB, since DB is the only register
that is guaranteed not to change. DL (which - since it, like all
registers save for DB itself, is expressed as a DB-relative address -
is a negative address) marks the lowest point in the stack that is
accessible to user code; it, too, can be moved to dynamically expand
or contract the useful DL-DB area, in which SL procedures (like
V/3000) can store global data.
What interests us is the "nether reaches," the area below DL. Your
mother may not have told you this, but there is a whole wealth of
information (for many programs I've seen, more interesting than the
stuff above DL) called the "PCBX" tucked away there. "PCBX" stands for
"PCB eXtension" (the PCB being an important system table that you'll
hear more about later), and contains much of the information that the
system needs to know about the process -- what files it has open, what
session it belongs to, where other tables that describe the session
more thoroughly (the JIT and JDT) are, what data segments this process
has allocated that must be deleted when it dies, and so on.
The PCBX is divided into four portions (see Ch.7 of Sys.Tables
manual):
- PXGLOB, which lists things like the capabilities of the person who
is running the process, the DST indexes (aka data segment numbers,
which we'll talk more about later) of the Job Information Table
(JIT) and Job Directory Table (JDT) pertaining to the process's
session, and so on.
- PXFIXED, which describes some other things about the process --
what data segments it has allocated, what session it belongs to,
what the values of its registers are (if it is not currently
active; if it is currently activate, i.e. being executed by the
CPU, the register values are actually stored in the CPU
registers), and so on.
- PXFILE, which describes the files that the process has opened.
- The pointer area, which points to the other portions. Unlike most
other pointers, these pointers are DL relative and are implied to
be negative; i.e. a value of 364 points to DL-364.
They are arranged roughly as follows:
DL > ---------------------------------------------
DL-1> | DL-relative pointer to PXGLOB | -------------->
DL-2> | DL-relative pointer to PXFIXED | ----------> |
DL-3> | DL-relative pointer to PXFILE | ------> | |
DL-4> | Special count that we don't care about | | | |
--------------------------------------------- | | |
| | | | |
| PXFILE | | | |
| | | | |
--------------------------------------------- <-----v | |
| | | |
| PXFIXED | | |
| | | |
--------------------------------------------- <---------v |
| | |
| PXGLOB | |
| | |
--------------------------------------------- <-------------v
Now, how does one access these system tables? Well, they are kept in
your stack, so they can be accessed using simple SPL DB-relative
pointers. For instance, to get word 2 of the PXGLOB (which happens to
be, in both MPE IV and MPE V, the zeroth word of the user's capability
mask, containing information on whether the user has SM capability,
AM, etc. -- all the capabilities save for PM, PH, DS, MR, IA, and BA)
you would merely have to do the following:
INTEGER POINTER DL;
INTEGER CAPS;
PUSH (DL); << push the address of DL onto the stack >>
@DL:=TOS; << DL now points to the DL register >>
GETPRIVMODE; << all DL-negative addressing must be done in priv >>
CAPS:=DL(
-DL(-1)+ << -DL(-1) is the DL-relative ptr to PXGLOB >>
2); << add 2 to get to the caps word >>
GETUSERMODE;
Voila! Note that we have to be in privileged mode to access ANY word
of the PCBX. After all, if you could do this in user mode, you could
change the line "CAPS:=DL(-DL(-1)+2)" to "DL(-DL(-1)+2):=CAPS" and
CHANGE the capabilities word instead of just reading it... in fact, a
variation of this is exactly what VESOFT's GOD program (which
temporarily -- for the duration of the session -- gives the user who
ran it all the capabilities and all the ALLOWs) does.
Similarly, one can get at all the other portions of the PCBX. For
instance, to get the Active File Table (AFT) entry for file number
FNUM, one can do the following:
INTEGER POINTER DL;
INTEGER ARRAY COPY'OF'AFT'ENTRY(0:5);
PUSH (DL);
@DL:=TOS;
GETPRIVMODE;
MOVE COPY'OF'AFT'ENTRY:=DL(-6*FNUM-4),(6);
GETUSERMODE;
Of course, you must first know that the AFT entry for file FNUM is 6
words long and always starts at location DL-6*FNUM-4 (in MPE V only --
this is different from MPE IV). Also note that the AFT is actually
known as either the Active File Table or the Available File Table
(depending on which chapter of the System Tables Manual you believe).
You can look up its format in chapters 6 and 7 of the manual.
If you want a brief exercise in using this access method, whip out
your trusty System Tables Manual and write a small program to print
out your current capability word (in octal) and your current
job/session number (HINT: it's in the PXFIXED). While you're at it,
also change your current capability word to give yourself SM
capability (make sure you do not already have it) and do a "LISTUSER
MANAGER.SYS" using the COMMAND intrinsic before and after this change.
You don't need to change it back since it is valid for the duration of
the process only, and when the process dies, the user is left with the
same capabilities as before. The solution to this problem (which we'll
call Problem 1) is in Appendix 1.
Of course, when you're learning to use these access methods, be sure
to run all your test programs during off hours or on a non-production
computer. I've crashed my share of computers when I was starting to
work with system tables, and you'll probably crash some, too.
Fortunately, once you get acquainted enough with the techniques
involved, and especially if you use some of the safety devices I'll
describe later, your programs will probably be far more reliable. In
the meantime, expect some foul-ups.
Another useful trick is using privileged mode DEBUG to ensure that
your program is working right. Say your program doesn't produce the
correct result -- it could either be because you're trying to get the
wrong value (e.g. PXGLOB location 3 doesn't really contain what you
think it contains) or that you're going for the right value, but your
method of getting it is wrong or you're outputting it wrong. To find
out what the problem is, you can use DEBUG to look at the value
manually -- if it's the expected value, then the bug is in your
retrieval or output algorithms; if it's not the expected value, you're
trying to retrieve from the wrong place.
The way to get at values in PXGLOB using privileged mode DEBUG (you
must be a privileged user to do this!) is to type a command of the
type "DDL-address,length". Don't forget that DEBUG by default assumes
octal input and output; to input a decimal number, prefix it with a
"#"; to make output come out in decimal, type "DDL-address,length,I".
Thus, to see the value of DL-1 (the PXGLOB pointer), type "DDL-1,1,I",
or just "DDL-1,I" since 1 is the default length. To find the value of
PXGLOB word 3, you can either first do a "DDL-1,I", and do
"DDL-#xxx+3,I" (where xxx is the value that the DDL-1 gave you), or
you can say "DDL-'DL-1'+3,I" -- the "'DL-1'" means "the value stored
at DL-1".
One important note: although the access method I just described works
well enough, I favor a different approach that I'll discuss when I get
to system tables kept in arbitrary data segments and the MFDS
instruction.
THERE'S DATA IN THEM THAR DATA SEGMENTS
If you peek into your friendly neighborhood System Tables manual
(which at last count had 23 chapters), you will find that about half
of chapter 7 is dedicated to tables that reside in stacks' DL-negative
areas, and you might start to wonder what the remaining 22 1/2
chapters are about. Well, most of the system tables described in the
manual are stored in operating system Data Segments, and if you
thought that DL-negative tables were tough to access, just wait until
you see these.
It doesn't make sense to keep data in fixed locations in memory, e.g.
putting the table of currently active jobs in locations %10400 through
%10777, a process's stack in locations %533000 through %534377, and so
on. For one, the very concept of "virtual memory" demands that the
operating system be able to swap data out from disk and then bring it
back into a different location in memory; furthermore, the HP3000 is
still a 16-bit machine (yes, Virginia, there are still 16-bit machines
out there, and you're using one of them), and it would have trouble
supporting addresses that are more than 16 bits long.
Rather, the data is broken up into many "segments". Within each
segment, all addresses are relative to the base of the segment; if one
segment has to point to another, it would have both the segment number
and the offset within the segment. This way, a segment can be
conveniently swapped in and out of memory, since the physical address
of the segment is kept in only one table that is managed by the
swapper; also, since most addresses are segment-internal, we can
usually get away with quick 16-bit addressing.
Consider for instance your own process stack -- it, too, is a data
segment. Addresses within it are relative to the process's DB register
(a slight difference from other segments, in which the addresses are
relative to the segment's base); the stack contains all your data plus
the tables in the DL-negative area that we talked about. In case you
should wish to access another process's stack, you can -- all you need
is its "data segment number." Similarly, the table of all the
currently active jobs (called the "JMAT," for Job MAster Table) is
also kept in its own data segment, as is the table describing all
currently active processes (the PCB -- Process Control Block). Any
such table can be accessed if you only know its data segment number.
Incidentally, let me mention a certain documentation inconsistencies
pertaining to data segments. What I call the "data segment number" is
sometimes called:
A "DST index." All existing data segments are described in a system
table (which itself is in a data segment) called the Data Segment
Table, a k a DST. The number of the data segment is thus the index
into that Data Segment Table.
A "DST number," a corruption of "DST index." Doesn't make much sense
-- Data Segment Table number?
A "DST," a further corruption of "DST index." Although this is
actually both ambiguous and technically improper, lots of people use
it (including yours truly). It's all HP's fault that they didn't
call it something decent like "data segment number" in the first
place.
I shall make a daring attempt to CONSISTENTLY call these critters data
segment numbers throughout this document, but don't count on it.
ACCESSING DATA SEGMENTS -- ONE APPROACH
We have established that most of the data worth knowing is kept in
data segments, and that a piece of data can be accessed by specifying
the data segment number and the offset within the data segment,
without caring about the physical memory address, which anyway will
probably change when the segment is swapped in and out. Now, the
question becomes: how do you get to it?
Well, thought some unknown HP designer when he was faced with making
all this up back in '72 when the 3000 was being built, we can already
access arbitrary DB-relative addresses in our stacks -- why don't we
just provide some way of switching the DB register to point to an
arbitrary data segment? So, say that someone wants to get location 3
of data segment 55 -- he'll just have to switch to segment 55, grab
location 3 (note that he'll do it just like he'd grab location 3 of
his own stack, except that now DB points to segment 55), and switch
back to his stack, the value of the desired location in hand.
Thus, our program would look something like:
<< ATTENTION: People who're just looking at the pictures:
THIS PROGRAM WON'T WORK! DON'T EVEN TRY IT! Read the text...
>>
$OPTION PRIVILEGED << PM or no dice! >>
BEGIN
INTEGER
DUMMY, << location DB+0 >>
I; << location DB+1 >>
<< This is a system procedure that switches DB to point to the
specified data segment, and returns the number of the
data segment to which it now points.
If the data segment number is 0, switches to the process's
stack. This is NOT the same as the documented privileged
procedure SWITCHDB -- THEY CAN NOT BE USED INTERCHANGEABLY!
SWITCHDB is only intended for accessing your own data segments
that were allocated by your program using GETDSEG. >>
INTEGER PROCEDURE EXCHANGEDB (DSEG'NUMBER);
VALUE DSEG'NUMBER;
INTEGER DSEG'NUMBER;
OPTION EXTERNAL;
DUMMY:=EXCHANGEDB (55);
ASSEMBLE (LOAD DB+3); << push the value at DB+3 onto the stack >>
I:=TOS; << pop it from the stack -- I now contains the value
of the 3rd location in segment 55 >>
DUMMY:=EXCHANGEDB (0); << switch back to the stack >>
<< now, do whatever we want to do to I >>
END.
What did we do? We set our DB register to point to data segment 55, we
got the value at location DB+3 -- now location 3 of segment 55 -- and
moved it into I, and then we switched back to our stack. Note that we
did not use the value returned by the first EXCHANGEDB, which is the
data segment number of the segment to which DB pointed to before --
namely, our stack -- as the data segment number in the second
EXCHANGEDB. That's because if we just EXCHANGEDB to our stack's data
segment number, DB will point to the BASE of the stack data segment;
however, specifying 0 as the data segment number is a special feature
that switches back to the process's stack data segment and makes DB
point to the right place.
But THIS DOESN'T WORK. What's more, IT FAILS IN A TRULY SPECTACULAR
WAY, most likely crashing your system or worse. Why? Because when we
do an EXCHANGEDB, ALL THE DB-RELATIVE ADDRESSES NOW POINT INTO THE
DATA SEGMENT, including all of our DB-relative variables -- such as I
-- and our arrays, either DB-relative (global), or Q-relative
(procedure local)! When we move the top of stack into I, we are moving
it to DB+1 -- the location in which I is stored -- and DB+1 now points
into the data segment, too! In fact, the only way in which the above
could be made to work is by making sure that whenever we use any
DB-relative addressing while in "split-stack mode" -- i.e. when we've
EXCHANGEDBd to another segment -- we mean addressing relative to the
data segment. For instance, we could leave the value of location 3 on
the stack, EXCHANGEDB back to our stack data segment, and then safely
pick the value up into I; or, we could put this in a procedure, and
make I a Q-relative variable. Remember, Q-relative and S-relative
addressing does not get changed; however, ALL DB-relative addressing
does.
The bottom line is that, for all practical purposes, we can have
access to only one segment at a time -- either our stack, in normal
mode, or one alternate data segment in split-stack mode. We can't
really work with both, except when we're in split-stack mode and we
use S-relative and Q-relative addressing. Remember, though that even
this is restricted since most arrays, even Q-relative ones, use DB
addressing anyway (because they have a pointer that points to the
data, and all these pointers are always DB-relative). So, if for the
entirety of our process's life we want to be dabbling with data
segment 55, we have no problem; however, if we usually want to work
with our stack -- remember, data segment 55 probably does not have
room for our temporary variables, arrays, and whatever else is usually
kept in our stack -- and only sometimes get at data segment 55, we
have a problem.
Thus, the rules of the game for split-stack mode operation are:
* Always be keenly conscious of when you are in split-stack mode.
Try to be in split-stack mode as little as possible (I prefer
never to be in split-stack mode except when some operating system
procedure that I'm calling -- such as DIRECSCAN, the directory
traversal procedure -- requires it).
* When in split-stack mode, remember that only:
- By-value procedure parameters;
- Simple (non-array) procedure-local variables;
- Variables explicitly declared to be Q-relative or S-relative
(using constructs like "INTEGER XYZZY = S-3;");
- Local arrays that are declared to be Q-relative (e.g. "INTEGER
ARRAY X(0:4)=Q;");
refer to data in your stack; all others refer to data in the data
segment to which you've switched.
A CIVILIZED ALTERNATIVE
Fortunately, there is another way of accessing data segments -- two
little-known privileged machine instructions called
MFDS (Move From Data Segment)
MTDS (Move To Data Segment)
Their principle of operation is simple -- they move data from the
stack to a data segment or from a data segment to the stack. Thus, if
you want to read location 3 of data segment 55, you'd issue an MFDS
instruction indicating that you want to move 1 word from location 3 of
data segment 55 to some buffer in your stack. It's simple, and since
you don't do an EXCHANGEDB, you can keep using all of your variables
with no problems.
Now, let's be a bit more specific: just how does one tell the
instruction where to move from and where to move to? Well, the best
place where instructions can take input parameters is from the stack
-- not just from your stack data segment, but from the topmost few
locations on the stack, just below the S register. You can put
parameters onto the stack using the SPL construct
TOS:=parameter value;
and then issue the instruction by entering
ASSEMBLE (MFDS 4);
or
ASSEMBLE (MTDS 4);
The "4" in the ASSEMBLE statement is the so-called "stack decrement"
-- it indicates how many of the parameters passed to the instruction
should be removed from the stack once the instruction is done. Each
instruction takes 4 parameters (which we'll talk about soon), and
there's no reason to leave any of them laying around on the stack, so
we tell the instruction to get rid of all 4 of them.
Now, each instruction must take as parameters:
The address of the buffer you want to move from or move to. Usually,
this is "@BUFFER", where BUFFER is the name of the buffer array.
This must be a word address; this usually means that if you specify
"@BUFFER", BUFFER must not be a byte array, but should rather be an
integer, logical, or double array.
The data segment number you want to move to or move from.
The offset in the data segment at which you want to start the move.
The length (in words) of the data to be moved.
Note that if you specify an incorrect data segment number, invalid
offset, invalid buffer address, or invalid length, HP will reward you
with a system failure...
The order of the parameters is NOT the same for both instructions.
Rather, for each instruction you should put onto the stack the
parameters that describe "where to move to", then the parameters that
describe "where to move from", and then the length. In other words,
the MFDS calling sequence would be
TOS:=@BUFFER; << move to >>
TOS:=DSEG'NUMBER; << move from >>
TOS:=OFFSET'IN'DSEG; << move from >>
TOS:=LENGTH;
ASSEMBLE (MFDS 4);
and the MTDS calling sequence would be
TOS:=DSEG'NUMBER; << move to >>
TOS:=OFFSET'IN'DSEG; << move to >>
TOS:=@BUFFER; << move from >>
TOS:=LENGTH;
ASSEMBLE (MTDS 4);
Also, don't forget that the instructions must be executed
when in privileged mode.
So, let's try a real-live application. Say that you want to write a
program that prints out the session limit -- the maximum number of
sessions that can be running at one time. Let's see how you could go
about doing it.
There's an old joke about a Chinese recipe for broiled rabbit -- the
recipe starts with "first, catch the rabbit". We must first find out
where the session limit is stored. Well, the session limit is
logically job information ("job" in this context pertains to both
batch jobs and online sessions), so it would most reasonably be in the
"JOB INFORMATION" chapter of the System Tables manual -- Chapter 8. In
fact, on page 8-??? is the layout of a table called the JMAT (Job
MAster Table), and lo and behold, there in word 8 (MPE IV) or in word
10 (MPE V), is the session limit. So far, so good. And, what's more,
on that very page (or barring that, in chapter 2) it says that the
JMAT is data segment number (they probably say "DST Entry
assignments") 25.
So, the session limit is word 8 (MPE IV) or word 10 (MPE V) of data
segment number 25. We know that it's one word long, so we can write
the following program:
$CONTROL NOSOURCE, USLINIT
<< :PREP me with ;CAP=PM >>
BEGIN
INTRINSIC
GETPRIVMODE,
GETUSERMODE,
QUIT;
INTEGER
SESSION'LIMIT;
GETPRIVMODE;
TOS:=@SESSION'LIMIT; << Move to the SESSION'LIMIT variable >>
TOS:=25; << Move from data segment 25 >>
TOS:=8; << MPE IV; in MPE V, use 10 >>
TOS:=1; << Move 1 word >>
ASSEMBLE (MFDS 4); << Move From Data Segment >>
GETUSERMODE;
QUIT (SESSION'LIMIT);
END.
When you run this program, it'll get the session limit, and call QUIT
with the specified parameter. I decided to call QUIT because QUIT will
print the parameter and it's easier than calling PRINT and ASCII (I'm
actually used to using my own SPL output package, which I described in
the "Winning at MPE" column of the DEC 1983 through MAR 1984 Interact
magazines; it makes numeric formatting much easier).
Now, you know for sure whether or not this program works because you
can do a :SHOWJOB and see the real session limit. However, if you
tried to move something from a data segment and wasn't sure whether
your program was working right or not, you could use privileged mode
DEBUG to check it out. The command you'd use is "DDA" (Display DAta
segment), and you'd enter "DDA dsegnumber+offset,length". Thus, to
check out the above program, you can type:
:DEBUG
*DEBUG* PRIV.xx.xx
?DDA#25+#8 <<or #10 for MPE V>>,1,I
DA31+10 +xxxxx
?E
The "xxx"s stand for the values that depend on your system
configuration -- the "+xxxxx" that comes out on the same line as
"DA31+10" (or "DA31+12" in MPE V systems) is your session limit. Note
that DEBUG echoes "DA31+10" -- 31 is the octal representation of
decimal 25, and 10 is the octal for decimal 8.
So that's really all there is to it -- figure out the data segment
number to which to move from/to (which is either a constant given in
the System Tables manual or a variable kept in some other system
table) and the address within the data segment to move from/to, and
then issue the appropriate MFDS or MTDS instruction. It's really
rather simple, and it's a shame that HP doesn't explain it anywhere
except a remote corner of the Machine Instruction Set Manual (and even
there, not too well). It would have made sense for HP to describe this
-- in fact, describe everything mentioned in this paper -- in the
System Tables Manual or even a more widely distributed document. Oh
well, I guess that's just not the "HP way".
However, there is one gigantic problem with the above approach -- even
if you're just doing MFDSs, passing an incorrect parameter will crash
the system. Even if you write a procedure that does the "TOS:="s and
the ASSEMBLEs, thus avoiding typographical errors, you're still going
to have many bugs in your program, and having the system crash for
each one of them is not a very appealing prospect (in fact, a very
appalling prospect).
This is especially a problem if you're not accessing permanent system
tables (like the JMAT) but rather more ephemeral tables like other
process's stacks. You could easily get the data segment number of
another process's stack, and by the time you try to read it, the
process (and the data segment) might be gone.
The solution is to write procedures that get passed the MFDS/MTDS
parameters, check them to ensure validity, and only do the MFDS or
MTDS if the parameters are OK. That's what I'm going to talk about
now, because writing code that uses MFDS and MTDS without having these
safeguards (at least while debugging) is, in my opinion, grossly
impractical.
What kinds of errors can you have in your choice of parameters? Well
one is that you're moving data from or into the wrong place. There's
nothing any automatic checking can do about this -- that's a logical
error that you'll have to find yourself. I only hope that these
logical errors will happen to you mostly on MFDSs and not on MTDSs.
There are several errors, though, that are automatically detectable:
Negative length, data segment number, or offset. I'm not certain
about this (and I'm not going to risk a system failure to try to
find out), but maybe a negative length would mean "right-to-left"
movement (like it does in the MOVE statement -- see the section on
the MOVE statement in the SPL Reference Manual). However, since even
if this were so, it wouldn't be a very useful option, a negative
length, data segment number, or offset are pretty certainly errors.
Note that negative buffer address is NOT an error -- remember,
buffers in the DL-to-DB area will have negative addresses. Also,
data segment number 0 is an error.
Invalid data segment number. Not all data segment numbers are valid
-- they can be up to 1023 in MPE IV and even more in MPE V, and
there aren't always going to be that many real, allocated data
segments. How could we check for this? Well, we'd have to look this
up in another system table, the Data Segment Table (DST); it's
described in System Tables Manual Chapter 2, and we'll talk more
about it later.
Buffer address out of bounds. This means that the starting address
of the buffer is less than the DL register (i.e. the buffer starts
in the PCBX or even lower, outside of your stack in somebody else's
chunk of memory) or the ending address of the buffer
(@BUFFER+LENGTH-1) is greater than the S register. Remember, this is
privileged mode -- no bounds violation checking!
Data segment offset out of bounds. This means that the offset is
less than zero (already mentioned above) or offset+length-1 is
greater than the current length of the data segment. The length of
the data segment is also stored in the DST.
So, our code would look something like:
$CONTROL NOSOURCE, SEGMENT=DSEG'IO, SUBPROGRAM, USLINIT
<< Caller must be :PREPped with ;CAP=PM >>
BEGIN
DEFINE
TURNOFFTRAPS =
BEGIN
PUSH (STATUS);
TOS.(2:1):=0;
SET (STATUS);
END #;
LOGICAL PROCEDURE DSEG'CHECKPARMS
(DSEG'NUMBER, OFFSET, BUFFER, LENGTH);
VALUE
DSEG'NUMBER,
OFFSET,
LENGTH;
INTEGER
DSEG'NUMBER,
OFFSET,
LENGTH;
ARRAY
BUFFER;
BEGIN
<< Returns TRUE if all parameters OK. >>
INTRINSIC
GETPRIVMODE;
INTEGER POINTER
DST = 2; << I'll talk more about this later >>
INTEGER POINTER
DL;
<< First, find out the address of DL -- the lowest valid stack
address -- for use later on. >>
PUSH (DL); << Push the value of the DL register onto the stack >>
@DL:=TOS; << Pop it into our own pointer called "DL" >>
<< Now, get privileged mode for subsequent uses of "DST", the
system table pointer we defined above (more about this later).
Privileged mode should NOT be relinquished by this procedure
(again, more about this later). >>
TURNOFFTRAPS;
GETPRIVMODE;
<< Now, check all the possible conditions >>
IF
<< Is any of the parameters negative (or DST=0)? >>
DSEG'NUMBER<=0 OR OFFSET<0 OR LENGTH<0
OR
<< Is DSEG'NUMBER greater than the maximum data segment #? >>
DSEG'NUMBER>=DST(0)
OR
<< Is DSEG'NUMBER invalid (indicated by having the length
in its DST entry be 0)? >>
4*DST(4*DSEG'NUMBER).(3:13)=0
OR
<< Is the last data segment address to be moved
(OFFSET+LENGTH-1) greater than the last valid data segment
address (which is (data segment length - 1))? >>
OFFSET+LENGTH-1>DST(4*DSEG'NUMBER).(3:13)*4-1
OR
<< Is the starting address of the buffer below DL? >>
@BUFFER<@DL
OR
<< Is the ending address of the buffer (@BUFFER(LENGTH-1)) above
the greatest possible address, which is just one word below
the first parameter to this procedure? >>
@BUFFER(LENGTH-1)>@DSEG'NUMBER-1
THEN
DSEG'CHECKPARMS:=FALSE
ELSE
DSEG'CHECKPARMS:=TRUE;
END; << the procedure exit gets you back to user mode >>
PROCEDURE DSEGREAD
(DSEG'NUMBER, OFFSET, BUFFER, LENGTH);
VALUE
DSEG'NUMBER,
OFFSET,
LENGTH;
INTEGER
DSEG'NUMBER,
OFFSET,
LENGTH;
ARRAY
BUFFER;
BEGIN
<< Reads LENGTH words from offset OFFSET of data segment #
DSEG'NUMBER into the array BUFFER.
Calls QUIT if any parameter is invalid. >>
INTRINSIC
GETPRIVMODE,
QUIT;
IF NOT DSEG'CHECKPARMS (DSEG'NUMBER, OFFSET, BUFFER, LENGTH) THEN
QUIT (1717)
ELSE
BEGIN
TOS:=@BUFFER;
TOS:=DSEG'NUMBER;
TOS:=OFFSET;
TOS:=LENGTH;
TURNOFFTRAPS;
GETPRIVMODE;
ASSEMBLE (MFDS 4);
END;
END; << the procedure exit gets you back to user mode >>
PROCEDURE DSEGWRITE
(DSEG'NUMBER, OFFSET, BUFFER, LENGTH);
VALUE
DSEG'NUMBER,
OFFSET,
LENGTH;
INTEGER
DSEG'NUMBER,
OFFSET,
LENGTH;
ARRAY
BUFFER;
BEGIN
<< Writes LENGTH words from the array BUFFER to offset OFFSET of
data segment # DSEG'NUMBER.
Calls QUIT if any parameter is invalid. >>
INTRINSIC
GETPRIVMODE,
QUIT;
IF NOT DSEG'CHECKPARMS (DSEG'NUMBER, OFFSET, BUFFER, LENGTH) THEN
QUIT (1718)
ELSE
BEGIN
TOS:=DSEG'NUMBER;
TOS:=OFFSET;
TOS:=@BUFFER;
TOS:=LENGTH;
TURNOFFTRAPS;
GETPRIVMODE;
ASSEMBLE (MTDS 4);
END;
END; << the procedure exit gets you back to user mode >>
END.
I'll be the first to admit -- that's a lot of code. But with it, I am
able to confidently test all my programs that read data segments
(those that write other data segments can, of course, cause other
nasty problems) on a production computer during working hours with no
fear at all. In fact, I don't recall a single system failure in my
past year of development, in which I've been writing and maintaining
many programs like VESOFT's MPEX, LOGOFF, and others, all of which do
heavy system tables work. When I did the conversion of these programs
from MPE IV to MPE V, the first thing I did was to ensure that
DSEGREAD works. Once that was done, despite the fact that there were
bugs in the programs that it took several hours to iron out, there
were no system failures.
Now, a couple of comments about the above code are in order:
* First, note that the procedures abort whenever an incorrect
parameter is passed -- why? Well, an incorrect parameter is almost
guaranteed to be caused by a program bug (by definition). What's
the use of going on, doing other things, when you know that due to
a problem bug, your program couldn't read or write data that it
may be relying on? You could, of course, have the procedure return
a logical flag, but then you'd have to check the flag every time
you call it and probably wind up calling QUIT anyway. A good idea,
however, is to somehow get the procedure to indicate the place
they aborted and the cause of their abort -- negative parameter,
invalid data segment number, bad offset, etc.
* Second, note that all three procedures get privileged mode AND
NEVER GET USER MODE! This is because whenever a procedure is
exited, the mode is always reset to the mode of the caller; so, we
don't need to get user mode explicitly. What's more, if we try to,
it could actually hurt because it will cause an abort if the
calling procedure itself is privileged. MPE does not permit code
executing in user mode to exit to code executing in privileged
mode. So, getting user mode explicitly is both unnecessary and
undesirable.
* Third, what's this "TURNOFFTRAPS" nonsense? Well, once upon a time
(a long, long time ago, in a galaxy far, far away but really in
Cupertino), I was having problems with my privileged code
aborting. Well, a friendly HP person suggested that I turn off
arithmetic traps (things like INTEGER OVERFLOW and others) before
going into privileged mode, and lo and behold! all was well.
Apparently, some privileged operations require arithmetic traps to
be off (although why I'll never know), so to be on the safe side,
I always turn them off before going into privileged mode. Like
privileged mode, an exit from a procedure resets the setting of
the arithmetic traps to what it was in the calling procedure. Note
that some people (including Stan Sieler, an ex-HPer now at Allegro
Consultants, who's likely to know about these things) believe that
you only need to TURNOFFTRAPS before calling a system internal
procedure (ATTACHIO, EXCHANGEDB, GETSIR, etc.) and not when you're
just executing a privileged instruction (MFDS, LST, etc.). They
may be right -- try it both ways and see.
* Finally, let me just caution you that the above checks are NOT
foolproof. Not only will they allow you to write incorrect data
into data segments -- they'll make certain that the data segment
number's correct but there's no way they could check the data --
but also, if you read or write a segment that is still around but
is being deleted (or contracted), the procedure might check the
segment while it's still around, see that all's well, and then do
the MFDS/MTDS after the segment is gone. In this case, the system
will indeed crash. I have NEVER gotten this to happen, but it's
possible -- the only way to completely avoid it is to make sure
that there are no bugs in your programs. Good luck...
USING MFDS/MTDS TO ACCESS THE PCBX AREA
When I was talking about accessing the DL-negative area, I mentioned
that there was a way of doing it that I liked better than the one I
presented. Well, here it is.
Remember, your stack is a data segment like any other segment. You can
access it using MFDS just as easily (or as difficultly) as, say, the
PCB, or somebody else's stack. All you have to do is find its data
segment number, which is not very hard, since it is in your process's
PCB (that's Process Control Block -- a very useful system table)
entry. Then, to read the PXGLOB, just do a DSEGREAD from your stack
segment, location 0, length 8 (MPE IV) or 12 (MPE V). To get the
PXFIXED, just do a DSEGREAD from your stack segment, with a starting
location of 8 (MPE IV) or 12 (MPE V) -- just above the PXGLOB. To get
the PXFILE, you have to get the PXGLOB, get the address of DL relative
to the start of the data segment (that's PXGLOB cell 0), and then get
the PXFILE address from DL-3.
So, just write the following procedure:
INTEGER PROCEDURE MYSTACK;
BEGIN
<< Returns the data segment number of the process's stack >>
INTRINSIC
GETPRIVMODE;
INTEGER POINTER << see below for more info on this construct >>
PCB = 3;
<< SL procedure that returns the process's PIN >>
INTEGER PROCEDURE MYPIN; OPTION EXTERNAL;
TURNOFFTRAPS;
GETPRIVMODE;
MYSTACK:=
<< MPE IV: >> PCB (16*MYPIN+3).(1:10);
<< MPE V: >> PCB (21*MYPIN+3).(2:14);
END;
Note the use of the MYPIN procedure -- that's how you get your Process
Identification Number (PIN), which is also the number of your PCB
entry.
Now, you can just call DSEGREAD, pass to it MYSTACK, and presto!
you're reading your own stack.
"Now, Eugene," you must be saying, "why would you want do a damn fool
thing like that? There you have the DL negative area, easy as pie to
access using pointers, and you insist on using an entirely different
strategy. Why bother?" Well, the answer is really quite simple -- I'm
lazy. I don't want to have to write and remember two sets of
procedures -- DSEGREAD/DSEGWRITE and PCBXREAD/PCBXWRITE. I'd rather
have one set that I would use to read either a data segment or the
PCBX, and one way of looking at things -- everything's just another
data segment, including the PCBX. System tables are complicated
things, and any bit of conceptual simplification helps.
AN EXAMPLE -- GETTING YOUR OWN SESSION NAME
The perfect example of using MFDSs (or actually, DSEGREAD -- you
should always call DSEGREAD, and never have an MFDS or MTDS anywhere
else in your code) is getting your own job/session name. This is a
useful piece of information that no HP-supplied intrinsic gives you --
you can use it to identify users (e.g. there's one user called
CLERK.PAYROLL, with session names JOE, SUSAN, etc. -- a technique we
support and encourage with our VESOFT SECURITY/3000 product), provide
accounting information, and whatever else.
First, we have to find out where it is. It comes under the broad
category of "JOB INFORMATION", and if you look into the System Tables
Manual, Chapter 8, you'll find it as a field of the Job Information
Table (also known as the JIT). But, the JIT isn't like other tables,
like the PCB, DST, or JMAT, which have constant data segment numbers
-- each session has its own JIT. Well, it turns out (and this is NOT
easy to find out) that the data segment number of the JIT is stored in
the PXGLOB. So, what you'd do is:
* Get the PXGLOB (using either of the methods shown above of
accessing your PCBX).
* With the JIT data segment number from the PXGLOB, get the JIT.
* Extract the session name from the JIT.
To see the actual program, look at program 2 in Appendix 1.
ACCESSING MEMORY-RESIDENT SYSTEM TABLES
As if EXCHANGEDB and MFDS/MTDS weren't confusing enough, there's yet
another way of accessing some system tables.
Certain system tables (e.g. the PCB, DST, CST [Code Segment Table],
etc.) are memory-resident -- they are always in main memory and are
never swapped out to disc. These tables can be accessed using two
special machine instructions called LST (Load from System Table) and
SST (Store into System Table). This would be a moot point if not for
the fact that SPL has an feature that uses these instructions to allow
you to easily access memory-resident system tables. (Note that this
feature has only been documented in the FEB 84 version of the SPL
reference manual, p. 2-13, 7-18, and 7-19; all prior versions of the
reference manual do not mention it.)
In SPL, if you say "INTEGER POINTER name = number;", all subsequent
references to "name(index)" will access the index'th word of the
memory-resident table indicated by "number". Thus, if you declare
"INTEGER POINTER TAB'PCB = 3;", saying "I:=2+TAB'PCB(1);" will set I
to 2 plus the value of word 1 of the PCB. Or, if you say
"TAB'PCB(SIZE'PCB*PIN+7):=24;", it will move 24 to the
SIZE'PCB*PIN+7th word of the PCB.
This means that to access these tables, you need not do an MFDS or
MTDS -- which are somewhat slower and also more cumbersome to call --
but rather just treat the table as if it were an array that you could
index just like an ordinary array.
Several things must be noted about these constructs:
* MEMORY-RESIDENT TABLE IDENTIFIERS (the numbers that you put after
the "=" in an INTEGER POINTER declaration) ARE NOT THE SAME AS
DATA SEGMENT NUMBERS! In the case of the PCB, the data segment
number and the memory-resident table identifier happen to be the
same (3). For other tables, this may not be the case. The way to
determine a table's memory-resident table identifier is to look
into Chapter 1 of the System Tables Manual where the System Global
(SYSGLOB) area is described. If the Nth word is indicated as
containing the address of (or pointer to) a system table, then N
is that table's identifying number.
* Memory-resident tables can only be accessed in privileged mode.
That means that whenever you want to use a memory table pointer
(declared using the "INTEGER POINTER name = number" construct),
you must be in privileged mode.
* Memory-resident table pointers can only be used when indexed, and
then only in expressions or in assignment statements. They can not
be used in MOVE or SCAN statements, as by-reference parameters in
procedure calls, or without an index. If you want to move several
words from a system table, use either an MFDS or memory-resident
table pointers and a FOR loop.
* I haven't the foggiest notion of what would happen if you specify
an invalid index when using a memory-resident table pointer (i.e.
a negative index or one that is greater than the size of the
table). I'd guess that if you pass an index greater than the table
size, it would just get you data from whatever happens to be in
memory at the system table base + the index, but if you specify a
negative index or an index that would cause the effective address
(the system table base + the index) to be outside of the memory
bank in which the system table is located, be prepared for a
system failure.
* Remember that relatively few system tables are accessible in this
way.
Personally, I do not often use this method of system table access. For
one, as I said before, I like to consistently use one approach, and
not confuse myself with many different ones -- my favorite is
MFDS/MTDS. Furthermore, since I have allocated a special array for
each system table, and have DEFINEs that access that array, I usually
end up having to copy an entire table entry anyway (not just a single
word); however, if you use the alternative approach described above
(i.e. having DEFINEs specify only the index into the entry without the
array name), you wouldn't have to move the entire table entry.
Sometimes, I do use memory-resident table pointers, primarily when
speed is of the essence -- I've been told that LST and SST are faster
than MFDS and MTDS. It's mostly a matter of personal preference -- use
whichever approach you find most appropriate.
An example of this approach could be the following procedure that
determines whether it's running on MPE IV or MPE V:
LOGICAL PROCEDURE MPE'V;
BEGIN
<< result := TRUE if MPE V, FALSE if MPE IV >>
INTRINSIC GETPRIVMODE;
INTEGER POINTER TAB'PCB = 3;
DEFINE PCB'ENTRY'LEN = TAB'PCB (1) #;
EQUATE MPE'V'ENTRY'LEN = %25;
TURNOFFTRAPS;
GETPRIVMODE;
IF PCB'ENTRY'LEN=MPE'V'ENTRY'LEN THEN
MPE'V:=TRUE
ELSE
MPE'V:=FALSE;
END;
Note that the PCB table entry size changed from %20 in MPE IV to %25
in MPE V. Since the entry size is stored (in both MPE IV and MPE V) in
word 1 of the PCB table, we can just look at that word and see if it's
%25 or not.
This procedure is exceptionally useful for writing programs that work
on either MPE IV or MPE V, or at least abort if they're run on a
version of MPE other than the one they were written for. If your
program is written for MPE IV, it's much better to abort with a nice
error message when run on MPE V rather than crashing the system.
Note however that if you wanted to, you could perform the same task
using MFDS.
ABSOLUTE MEMORY LOCATIONS
Some data is stored not in data segment-relative locations, but rather
absolute memory addresses. For instance, there is an area in memory
called "SYSGLOB" (which stands for SYStem GLOBal), which contains
certain interesting pieces of information, like the current console
device, the global allow mask, and lots of other goodies. It happens
to be stored at a fixed address -- it goes from locations %1000 to
%1377 in memory bank 0, and can be accessed using the "ABSOLUTE"
construct of SPL. For instance to determine the system console logical
device number (which is stored in location %74 of the SYSGLOB), you'd
do something like:
TURNOFFTRAPS;
GETPRIVMODE;
CONSOLE'LDEV:=ABSOLUTE (%1074);
GETUSERMODE;
TURNONTRAPS;
Simple -- think of "ABSOLUTE" as an integer array whose 0th word is
memory address 0 of bank 0. To store a value into this location, just
say "ABSOLUTE (%1074) := NEW'CONSOLE'LDEV;". Note however that unlike
a normal array, you can only read and write individual absolute
locations; you can't do a MOVE or SCAN with an absolute address, nor
can you pass it by reference to a procedure. In this respect, it's
like the system table pointers discussed above.
To confirm your results, or to access absolute locations without a
program, you can use privileged mode DEBUG's "DA" (Display Absolute)
command:
:DEBUG
*DEBUG* PRIV.xx.xx
?DA1074,I
A1074 +00020
?E
Also note that DEBUG has a "DSY" (Display SYstem global) command, with
"DSY n" being equivalent to "DA 1000+n" -- it just gets you the nth
word of SYSGLOB.
One word of caution: ABSOLUTE can only be used to access memory
locations with addresses 0 to 65535 -- those memory locations in the
so-called "memory bank 0". HP has recently been on a campaign to
reduce the amount of data that must be stored in bank 0 -- tables like
the PCB, CST, DST, and others that used to be always in bank 0 in MPE
IV are no longer always in bank 0 in MPE V. The upshot of this is that
since the only things that can be accessed using ABSOLUTE are the ones
that are guaranteed to be stored in bank 0, you should avoid using
ABSOLUTE whenever a non-bank 0 dependent access method (e.g. MFDS) is
available.
DISC RESIDENT SYSTEM TABLES, PART I -- FILE LABELS
Unlike process information, data segment information, and job
information, which are stored in memory, file information -- the size
of a file, its file code, lockword, location on disc, etc. -- can not
be stored in memory because it has to stay around through system
failures.
Most of the important file information, including all of the stuff
that all modes of :LISTF show, is stored on disc in "file labels". A
file's file label is pointed to by its directory entry and occupies
one sector (128 words) on disc. Since it's stored in disc rather than
in memory, accessing it is rather different from accessing data
segments.
In order to access -- read or write -- a file's file label you must
first find out the logical device number (LDEV) of the disc on which
the file label resides and the sector number of the file label on that
disc. This is by no means a trivial task, and in many instances is a
lot more complicated than actually accessing the file label once you
have this. Logical device numbers and sector addresses are stored in
various system tables, in a variety of formats (which we'll discuss
later). However, for the duration of this discussion, we'll assume
that you are trying to read or write the file label of a file that you
have opened; that way, you can figure out the file label address by
calling FGETINFO (or FFILEINFO mode 19).
Now you have the file label address -- a doubleword, containing the
LDEV in the 8 most significant bits and the sector number in the 24
least significant bits (a format we'll call type L). All you need to
do is to perform the actual I/O.
The fundamental file label I/O procedure is, not surprisingly, FLABIO:
INTEGER PROCEDURE FLABIO (LDEV, ADDRESS, CMD, FLAB);
VALUE LDEV, ADDRESS, CMD;
INTEGER LDEV, CMD;
DOUBLE ADDRESS;
INTEGER ARRAY FLAB;
OPTION EXTERNAL, UNCALLABLE; << must be called from PM >>
Its operation is quite simple -- you pass to it the LDEV and sector
address, CMD=0 to read or CMD=1 to write, and the 128-word array into
which the file label is to be read or from which it is to be written.
The result returned is 0 if all is OK, 1 if there was a "soft error"
doing the I/O (I think that this means that the check sum in the file
label being read is wrong), and 2 if there was a "hard error" doing
the I/O (I think that this means a physical I/O error). Simple enough,
once you have the file label's address.
So, we can write the following procedure:
<< Read the file label of the file opened as FNUM into the array
FLAB; return true if all OK, FALSE if failed. >>
LOGICAL PROCEDURE FREADFILELABEL (FNUM, FLAB);
VALUE FNUM;
INTEGER FNUM;
ARRAY FLAB;
BEGIN
INTRINSIC
FGETINFO,
GETPRIVMODE;
INTEGER LDEV;
DOUBLE ADDRESS;
INTEGER ARRAY ADDRESS'I(*)=ADDRESS;
INTEGER PROCEDURE FLABIO (LDEV, ADDRESS, CMD, FLAB);
VALUE LDEV, ADDRESS, CMD;
INTEGER LDEV, CMD;
DOUBLE ADDRESS;
INTEGER ARRAY FLAB;
OPTION EXTERNAL, UNCALLABLE;
<< To be really safe, you should have code here that checks to make
sure that reading 128 words into FLAB won't cause a bounds
violation. For more info on this, see below in the discussion
of disc address validity checking. >>
FFILEINFO (FNUM, 19, ADDRESS);
IF <> OR ADDRESS=0 THEN
<< either file not opened or not on disc (address=0) >>
FREADFILELABEL:=FALSE
ELSE
BEGIN
LDEV:=ADDRESS'I(0).(0:8); << high-order 8 bits >>
ADDRESS'I(0).(0:8):=0; << ADDRESS now is sector address >>
TURNOFFTRAPS; << See above (MFDS) to learn about this >>
GETPRIVMODE;
IF FLABIO (LDEV, ADDRESS, 0 << read >>, FLAB) <> 0 THEN
FREADFILELABEL:=FALSE
ELSE
FREADFILELABEL:=TRUE;
<< no GETUSERMODE -- also see above to learn about this >>
TURNONTRAPS;
END;
END;
Using this procedure, you can now read the file label of any file you
have opened, and find out a lot of information that FGETINFO and
FFILEINFO won't give you -- things like the file creation date, last
access date, last modification date, the file's extent map, lockword,
and other useful things. A similar procedure can be written to write
out an open file's file label -- just be sure that you don't use to
change things that are also kept in memory-resident file tables that
exist for open files (like the FCB, ACB, etc. -- see the System Tables
Manual).
Similarly, say that you want to read the file label of $OLDPASS
*without* opening it. Why would you want to? Well, in the case of
$OLDPASS you usually wouldn't (except if you want to save time) --
however, you might want to do this for other files that you can't open
so easily, like spool files or other people's temporary files or
$OLDPASSes.
The first thing you'd have to do is to find out where the file label
address for $OLDPASS is located. When you can recite this from memory,
you are truly a superior system programmer -- mere mortals would have
to scan through the System Tables Manual, make an educated guess or
two, and find it in the Job Information Table (chapter 8), words 36
and 37. Looks simple enough -- there is your file label address; all
you have to do is extract the LDEV from the 8 high-order bits, the
sector address from the remainder, and call FLABIO. Right? Wrong.
The fundamental means of doing the I/O are still the same -- you get
the LDEV, get the sector address, and call FLABIO. However, what is
stored in the high-order 8 bits of words 36 and 37 of the JIT is NOT
(!!!) the logical device number of the disc. The $OLDPASS address in
the JIT is what I call the "type V address" (as opposed to the "type L
address" described earlier) -- its high-order 8 bits are the so-called
"volume table index" of the disc on which the sector address is
contained.
So, you have a problem. You are trying to get the logical device
number of the disc on which the file label is located and the file
label's sector address. You have the file label's sector address, but
instead of the LDEV, you have a volume table index (VTABX).
What you need to do is to convert the VTABX into LDEV, and to do this
you use a system internal procedure called LUN:
INTEGER PROCEDURE LUN (VTABX, MVTABX);
VALUE VTABX,
MVTABX;
INTEGER VTABX,
MVTABX;
OPTION EXTERNAL, UNCALLABLE;
What LUN (which I suspect stands for "Logical Unit Number", another
name for LDEV) does is take a VTABX and a MVTABX (Mounted Volume Table
Index) and return the LDEV which they describe. Where do you get the
MVTABX? Well, in this case, it is also (fortunately) stored in the
JIT, in word 57.
So, to read $OLDPASS's file label, you'd do the following:
* First, read the JIT into an array (you can use the DSEGREAD
procedure we described earlier to do this).
* Then, take the $OLDPASS type V address (words 36 and 37), and
separate out the VTABX (high-order 8 bits) and the sector number
(low-order 24 bits).
* If $OLDPASS's address is 0, GO NO FURTHER -- it means that no
$OLDPASS file exists.
* Call LUN, passing to it the VTABX and the MVTABX (from word 57 of
the JIT).
* Make a prayer that you did everything right ("Oh Lord, watch over
this humble program and prevent it from crashing the system").
* Call FLABIO, passing to it the LDEV you got from LUN and the
sector address you got from the low-order 24 bits of words 36 and
37 of the JIT.
* I don't suppose I need to tell you to experiment with CMD=0
(read), not CMD=1 (write)...
Voila! All you ever wanted to know about FLABIOing in ten pages or
less...
Incidentally, you're probably wondering what all this VTABX and MVTABX
nonsense is about. Well, it was implemented to support Private
Volumes, in which the logical device number of a disc is not known
until the disc is actually mounted -- thus, the directory on the disc
would have to contain not LDEVs, but VTABXs. If you expect your
program to NEVER need to be run on Private Volumes, you can forget
about MVTABXs, and just call LUN with an MVTABX of 0. However, you
still have to call LUN, because even if your system never even smelled
a private volume, the VTABXs need not correspond to the LDEVs.
If you intend to use FLABIO often in your programs, I would suggest
that you write the following procedures:
* FLABREAD and FLABWRITE, which take a type L (LDEV and sector)
address, and either read into or write from a user-specified
array. For consistency's sake, they should return a file system
error number (e.g. FLABIO error 1 would map into file system error
108 [INVALID FILE LABEL]; error 2 would map into file system error
47 [I/O ERROR ON FILE LABEL]). Also, since passing an out-of-range
LDEV or sector to FLABIO will cause a system failure, this
procedure can check the address using some tricks I'll talk about
presently.
* V'TO'L'ADDRESS, which takes a type V (VTABX and sector) address
and a MVTABX, and returns the type L address. This would call LUN
to convert the VTABX and MVTABX into the LDEV, and would allow you
to easily read or write a file label given a type V address:
FLABREAD (V'TO'L'ADDRESS (V'ADDRESS, MVTABX), FLAB);
* FNUM'TO'L'ADDRESS, which takes the file number of an open file and
returns the type L address. Simply calls FFILEINFO or FGETINFO.
Again, makes reading/writing file labels given an FNUM easier:
FLABREAD (FNUM'TO'L'ADDRESS (FNUM), FLAB);
* Finally, FILE'TO'L'ADDRESS, which takes a filename and returns the
type L address. The simplest way to do this is to FOPEN the file,
call FNUM'TO'L'ADDRESS, and FCLOSE the file. A far more difficult
approach, which however is faster and doesn't care about file
security, lockwords, or whether the file is already opened, is to
get the file label address directly from the directory. For this,
you'd have to call the DIRECFIND procedure to get the sector
address from the directory, call DIRECFIND again to get the
directory entry for the group in which the file resides (to get
the group's MVTABX), and then call V'TO'L'ADDRESS to convert the
type V address that is stored in the directory to a type L
address. For the beginning, stick to the simple approach -- the
directory is probably worth a whole paper dedicated to it alone,
and will not be elaborated further in this document.
The algorithm of checking a type L address for validity is not
trivial, but still well worth implementing (unless you want to get a
reputation as "Mike 'System Failure' Johnson"):
* Call the CHECKDISC procedure:
PROCEDURE CHECKDISC (LDEV, STATUS);
VALUE LDEV;
INTEGER LDEV;
LOGICAL STATUS;
OPTION EXTERNAL, UNCALLABLE;
Pass to it the LDEV, and make sure that the low-order 3 bits (bits
.(13:3)) of STATUS are all 0 -- if bit 15 is set, this means the
LDEV is out of range; if bit 14 is set, the LDEV is not
configured; if bit 13 is set, the LDEV is not a disc. If none of
these bits is set, you know you have a good LDEV.
* Call the DISCSIZE procedure:
DOUBLE PROCEDURE DISCSIZE (LDEV);
VALUE LDEV;
INTEGER LDEV;
OPTION EXTERNAL, UNCALLABLE;
Pass to it the LDEV, and make sure that the sector address is
greater than or equal to 0 and less than the number DISCSIZE
returned to you (which is the number of sectors on the device).
* If you're a general-purpose procedure, check the buffer address
passed to you to make sure that it's within bounds, i.e. that its
start is at DL or above, and that its 127th word is at Q or below.
Remember that bounds violations are not checked for in privileged
mode, and trying to write to an out-of-bounds address would cause
whatever is at that address (i.e. in someone else's space) to be
over-written.
* If either check failed, abort -- you've got a bad address, which
if used in an FLABIO call would crash the system.
If you want to check the results of your programs, or dabble with file
labels without writing a program, there are several utilities you can
use:
* For some specific file label retrieval and modification tasks,
VESOFT's own MPEX, which allows you to easily look at files'
creator ids, access/modify/creation dates, etc., and modify many
file attributes (like creator id).
* HP's DISKED2, which allows you to read and write arbitrary disc
locations. If you intend to modify file labels, make sure that you
are either using the ">FILE" command of DISKED2 to get to the file
label's sector (rather than just a >DISC and >MODIFY) or set the
file label's checksum (word 34) to 0. If you use >DISC and >MODIFY
but do not set the checksum to 0, the file system will not update
the checksum and will think that the file label is corrupted.
* Privileged mode DEBUG, which has a "DV" command (Display Virtual
memory, a misnomer -- really displays an arbitrary disc sector).
Note that the syntax of this command, especially when the sector
address is more than 32768, is non-trivial; see the DEBUG Manual.
* FLUTIL3, a contributed program which allows you to read and write
the file label. It's easier to use than DISKED2, and is also safe
for modifying file labels.
Finally, a caveat to the user who knows about and uses ATTACHIO:
resist the temptation to do file label I/Os using ATTACHIO instead of
FLABIO. On reads, calling FLABIO is better because it checks the file
label's check sum, letting you know if the file label appears
corrupted. On writes, it is IMPERATIVE that you call FLABIO, because
otherwise the check sum will not be updated, and next time the system
wants to read the file label, it will think that it's corrupted.
As a final present to all you file label hunters out there, the
following table should tell you how to get file addresses:
* Permanent files -- FOPEN the file and call FGETINFO/FFILEINFO
(type L address); if you want to be fancy, call DIRECFIND to get
the file's directory entry (type V address).
* Your session's temporary files -- FOPEN/FGET(FILE)INFO (type L
address), or get type V address from your JDT (described in System
Tables Manual Chapter 8; pointed to by the PXGLOB, which is also
described in Chapter 8).
* Your session's $OLDPASS -- FOPEN/FGET(FILE)INFO (type L address),
or get type V address from your JDT (described in System Tables
Manual Chapter 8; pointed to by the PXGLOB, which is also
described in Chapter 8).
* Other sessions' temporary files and $OLDPASSes -- type V addresses
stored in each session's JIT or JDT.
* Spool files -- IDD or ODD system tables (System Tables Manual
Chapter 14); type L.
Good luck, and happy hunting.
DISC RESIDENT SYSTEM TABLES, PART II -- USING ATTACHIO
Other information besides file labels is also stored on discs --
things like disc labels, defective track tables, the contents of files
themselves, and so on. To get at them, you have to use a privileged
system internal procedure called ATTACHIO.
ATTACHIO is *the* primitive I/O procedure. You can use it to do any
arbitrary operation against any arbitrary device, from reading and
writing discs to reading, writing, and doing various control functions
to non-disc devices. All file system I/O functions eventually end up
as ATTACHIO calls.
The calling sequence for ATTACHIO is:
DOUBLE PROCEDURE ATTACHIO (LDEV,QMISC,DSTX,ADDR,FUNC,CNT,P1,P2,FLAG);
VALUE LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG;
INTEGER LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG;
OPTION EXTERNAL, UNCALLABLE;
This is quite a mouthful (just what *is* a QMISC?). Fortunately,
however, for discs these parameters are rather simple:
* LDEV: This is self-explanatory. I'm sure I need not tell you that
passing an incorrect LDEV will cause a system failure (#206, to be
precise).
* QMISC: 0. QMISC could have a lot of different values for non-disc
devices, but for disc devices 0 will do quite well.
* DSTX: Technically, the data segment number of the segment to/from
which the read/write is to take place. 0 means your stack, and
this is the value you'd use most often.
* ADDR: The address (within the data segment indicated by DSTX)
to/from which the read/write is to take place. When DSTX = 0 (i.e.
your stack), this is the ordinary word address of the array which
you're using for your I/O. Don't forget to have room for as much
as you want to read/write in the array, since like all other
privileged internal procedures, this one doesn't do ANY bounds
checking. Also note that if you're reading into a stack array, you
should be sure to pass the ADDRESS of the array (i.e.
"@arrayname"). "ATTACHIO (1,0,0,@FOO,0,10,ADDR'HI,ADDR'LO,1)" will
read into the array FOO; if you omit the "@" before the FOO, you
will read into the stack address pointed to by the 0th word of FOO
(which is probably not what you want).
* FUNC: Curiously enough, the function. 0 means read, 1 means write.
Try something else. I dare you.
* CNT: If positive, this is the number of words to read or write; if
negative, the absolute value of this is the number of bytes to
read or write. Thus, 10 means 10 words; -20 means the same thing
(20 bytes).
* P1: The high order word of the sector number for the I/O. Don't
forget that all reads and all writes must start at a sector
boundary (although CNT need not be a multiple of the sector size).
* P2: The low order word of the sector number. If you have a double
word address D, you can say
INTEGER D0=D, D1=D+1;
D0 will now refer to the high-order word of D and D1 to the
low-order word of D. Or you can calculate the high-order word of D
on the fly by saying "INTEGER(D&DLSR(16))" and the low-order word
by saying "INTEGER(D)".
* FLAG: 1. This means "blocked, wait until request is complete" --
the I/O system's equivalent of ordinary file I/O (as opposed to
no-wait I/O).
* Function returns a double-word:
Word 0 bits 8:5: qualifying status for the I/O; this further
describes the result of the operation. The general status (see
below) is usually all you need to find out what happened.
Word 0 bits 13:3: general status of the I/O:
0: Pending. I don't think this should ever happen with
flags = 1.
1: Normal.
2: End of file. I can't see how this could happen on disc
(remember, ATTACHIO knows nothing about disc files).
3: Unusual condition. The qualifying status probably describes
this better -- try figuring it out.
4: Irrecoverable error. Again, look at the qualifying status
if you can figure it out.
Word 1: number of words (positive) or bytes (negative)
transferred.
Incredibly, that's all there is to it. Once you've got an LDEV and a
sector address, just plug them into the right parameters, and call
away! Of course, remember to enter privileged mode and turn off traps
(just as you would with FLABIO or an MFDS/MTDS). Also, remember that
many disc addresses (the type V addresses I mentioned in the ACCESSING
FILE LABELS CHAPTER) contain not an LDEV, but rather a VTABX which
you'll have to convert into an LDEV. But, once you get all these
preliminaries out of the way, calling ATTACHIO is just as easy as
calling FLABIO.
For instance, say that we want to determine the number of defective
tracks on a given LDEV.
First, as always, we have to find out where this number is stored.
System tables Chapter 3 contains a lot of useful information on disc
format, including the fact that sector 1 of every disc contains the
disc's defective track table, and word 0 of the sector contains the
actual number of defective tracks.
Thus, our program would look something like:
INTEGER PROCEDURE NUM'DEF'TRACKS (LDEV);
VALUE LDEV;
INTEGER LDEV;
BEGIN
INTRINSIC GETPRIVMODE;
EQUATE SIZE'DEF'TRK'TAB = 128;
ARRAY DEF'TRK'TAB (0:SIZE'DEF'TRK'TAB-1);
DEFINE TAB'NUM'DEF'TRACKS = DEF'TRK'TAB (0) #;
INTEGER DISC'STATUS;
DEFINE LDEV'IS'DISC = (DISC'STATUS.(13:3) = 0) #;
EQUATE FUNC'READ = 0; << ATTACHIO read function >>
EQUATE DISC'IO'FLAGS = 1; << ATTACHIO flags for a disc I/O >>
PROCEDURE CHECKDISC (LDEV, STATUS);
VALUE LDEV;
INTEGER LDEV;
LOGICAL STATUS;
OPTION EXTERNAL, UNCALLABLE;
DOUBLE PROCEDURE ATTACHIO
(LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG);
VALUE LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG;
INTEGER LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG;
OPTION EXTERNAL, UNCALLABLE;
TURNOFFTRAPS; << See MFDS chapter for a discussion of this >>
GETPRIVMODE;
CHECKDISC (LDEV, DISC'STATUS);
IF NOT LDEV'IS'DISC THEN
NUM'DEF'TRACKS:=-1 << error, not a valid sic >>
ELSE
BEGIN
ATTACHIO (LDEV, <<qmisc>> 0,
<< Amazingly dreadful things will happen if we leave
off the "@" in "@DEF'TRK'TAB >>
<<dstx>> 0, <<address>> @DEF'TRK'TAB,
FUNC'READ, <<cnt>> SIZE'DEF'TRK'TAB,
<<sector address:>> 0, 1,
DISC'IO'FLAGS);
NUM'DEF'TRACKS:=TAB'NUM'DEF'TRACKS;
END;
END;
That's all. Relatively simple, and because of the CHECKDISC call that
ensures the LDEV is in range, configured, and a disc, no danger of
system failure.
Of course, this particular ATTACHIO calling sequence is only relevant
to discs; terminals, printers, tape drives, and other devices have
their own parameters for QMISC, FUNC, P1, P2, and FLAGS, which I will
not explain here for lack of space and because they really are not
relevant to system table access.
SIRS, AND WHAT HAVE THEY DONE TO GET SUCH DEFERENCE?
Whenever you are accessing (reading or writing) a system table that
somebody else might be modifying, you have to beware of something
being changed out from under you.
For instance, say that you're reading the ODD -- the Output Device
Directory, a rather odd name for the table that contains information
on all the output spool files in the system. The ODD is structured as
several linked lists of entries, one for each spooled device or device
class. The way that you'd read it is that you'd read one entry,
process it, get the address of the next entry from the current entry,
get the next entry, process it, and so on. However, say that while
you're processing this entry, the next entry in the list gets deleted.
Then, when you try to get the next entry from the location that is
pointed to by the current entry, you'll get garbage.
Even worse, say that you read the entry, process it, make some changes
to the copy of the entry you keep in your stack, and then try to write
it back out. Then, if the entry is deleted between the time you read
it and write it back out, you stand the risk of over-writing whatever
new entry might have been put there, probably with disastrous
consequences.
This is by no means a new problem -- it arises in file and database
systems all the time. The general solution to this is some kind of
locking mechanism, and that is precisely what SIRs are for.
SIR stands for System Internal Resource. You may lock it by calling
the privileged system internal procedure GETSIR and release it by
calling the privileged system internal procedure RELSIR. Since all (in
theory) processes that modify a system table must lock its SIR before
starting the modification, if you lock the SIR you are guaranteed that
no other process will modify the table until you unlock it.
Most system tables that are usually modifiable by more than one
process at a time (including things like the PCB, JMAT, ODD, etc., but
excluding JITs and JDTs, because these are session-local tables and
are thus very rarely concurrently accessed) have a SIR. The SIR
assignments are listed in Chapter 5 of the System Tables Manual.
Before using GETSIR and RELSIR, you have to declare them in your
program as follows:
INTEGER PROCEDURE GETSIR (SIR'NUMBER);
VALUE SIR'NUMBER;
INTEGER SIR'NUMBER;
OPTION EXTERNAL, UNCALLABLE;
PROCEDURE RELSIR (SIR'NUMBER, GETSIR'RESULT);
VALUE SIR'NUMBER, GETSIR'RESULT;
INTEGER SIR'NUMBER, GETSIR'RESULT;
OPTION EXTERNAL, UNCALLABLE;
When you want to get a SIR, you figure out its SIR number and pass it
to GETSIR, saving the result returned by GETSIR. Then, to release it
you call RELSIR, passing to it the SIR number and the result returned
by GETSIR (this is very important!). For instance, say you want to go
through the ODD, which is a linked list of entries, without being
afraid that somebody might be in the middle of adding or deleting an
entry, causing you to encounter a bad link. What you'd do is:
* Find out the ODD SIR number from the System Tables Manual Chapter
5 (it's 4).
* Lock the ODD SIR by calling GETSIR, saving the result in some
variable -- have a special variable, reserved for this purpose
(say, ODD'GETSIR'RESULT), that you are certain will not be
accidentally changed -- I and J are definitely out of the
question.
* Do whatever you want to do with the ODD, remembering that if you
abort for any reason before you unlock the SIR, THE SYSTEM WILL
CRASH. Also, be sure that neither you nor any procedure you might
call while you have the SIR tries to get a SIR whose priority
number (see the discussion below) is lower than the ODD's.
* When you're done, call RELSIR, passing to it the ODD SIR number
and the value that GETSIR returned.
* Breathe a sigh of relief that you haven't crashed the system.
Similar procedures are used for locking other SIRs -- however, note
that if you have more than one SIR locked at the same time, you must
unlock them in the REVERSE ORDER of locking. Also note that you won't
get any problems if you try to lock a SIR twice -- say, you lock it,
and another procedure you call locks it again; just remember to unlock
it twice, too (in the case both you and another procedure locks it,
all that is necessary is that both the other procedure and you unlock
it, again in the reverse order that you two locked it in).
Watch out, though -- when you're dabbling with SIRs, there are a lot
of possible pitfalls involved:
* Locking a SIR is not just a nice thing to do that'll avoid
problems for you. If you modify a table without first locking its
SIR, you get everybody else in deep trouble, much like if you were
modifying a shared file without first FLOCKing it. Unfortunately,
unlike KSAM files and IMAGE databases (but like MPE flat files),
no checking is done to ensure that you've gotten the right SIR --
it's your responsibility, and if you don't live up to it, may God
have mercy on your system.
* Like other resources, it's very easy to get into deadlock trouble
when you're getting more than one SIR. Say that you get SIR A and
then try to get SIR B. SIR B is locked by another process, so
you're suspended until the other process unlocks the SIR.
Meanwhile, the other process tries to get SIR A, and suspends
waiting for you to unlock it! Both of you are waiting for each
other to release a SIR, and you'll stay that way until the system
is rebooted. What's worse, anyone else who tries to lock either
SIR will be suspended, too, causing an ever-growing number of
suspended problems, and making the system come to a grinding halt.
To avoid this, there is a certain very fixed and unalterable SIR
locking sequence that you MUST obey. It is described in Chapter 5
of the MPE V System Tables Manual, where each SIR is given a
priority number, and you must never lock a SIR whose priority
number is lower than that of one you already have locked. Note
that a similar table in Chapter 5 of the MPE IV System Tables
Manual is quite incomplete -- try to get the MPE V locking
sequence (which, to the best of my knowledge, is the same as that
for MPE IV, although it is described much better). When you're
debugging SIR code that you're afraid might deadlock, it's a good
idea to keep an OPT or a SOO process running on some other
terminal. OPT and some versions of SOO have commands that allow
you to see who is holding what SIRs -- this can help you find the
cause of the deadlock.
* Not only must you watch out to make sure that you lock all SIRs in
the right order, you must also be certain that all your callers
and called procedures do too. Thus, if you want to lock the FILE
SIR (which must never be locked before the FMAVT [File
Multi-Access Vector Table] SIR), you must be sure that you do not
call any procedure that tries to lock the FMAVT SIR, since that
would be a SIR locking sequence violation. Since virtually all
file system intrinsics can under some conditions try to lock the
FMAVT SIR, you must either lock neither the FILE SIR nor the FMAVT
SIR or both the FILE SIR and the FMAVT SIR before calling a file
system intrinsic. That goes for all other procedures you call --
you better know what SIRs they try to lock, and you better make
sure that your SIR locking sequences are appropriate. Similarly,
if a procedure that you write uses SIRs, you have to be sure that
all your callers realize this and make sure that they either don't
have any SIRs locked when they call you, or they are certain that
your and their locking sequences mesh well.
* If your process terminates itself for any reason (TERMINATE, QUIT,
STACK OVERFLOW, whatever) while you have a SIR locked, you get a
System Failure 314. You best be VERY careful. Note that you need
only be afraid of your process terminating itself -- while you
hold a SIR, you can't be killed from outside (say, by an
:ABORTJOB); if someone tries, you'll keep on processing until you
release your last SIR, and then you'll die.
* Always disable break before locking a SIR. This is not because you
can be :ABORTed from break -- if someone tries, you'll keep on
going until you release your SIRs, and only then will the abort
take hold. Rather, if you let a user hit break while you've got a
SIR, the user might try to execute an MPE command that tries to
get the same SIR, and you'll get a deadlock -- the command waiting
for the SIR (which is held by your program) and your program
waiting for MPE to :RESUME it (which it can't because it's waiting
for a SIR). Deadlocks work in mysterious ways.
* Never do any terminal I/O while you have a SIR locked. In fact, do
not perform any task that you suspect might take a long time to
finish -- like issuing a request for a tape, reading a possibly
empty message file, etc. The reason for this is simple -- if a
process that has a SIR suspends, everybody else who wants the SIR
will suspend, too, thus effectively stopping the system until the
SIR holder is re-activated. The last thing you want is for the
system to come to a grinding halt because someone went on a coffee
break while a SIR-holding program was expecting input from him. As
I said, this is also relevant for terminal I/O because a control-S
on the terminal will suspend the writing program until a control-Q
is hit.
* Always be sure that the GETSIR result you pass to a RELSIR is the
result returned by the GETSIR that got the particular SIR
involved. If not, three guesses as to what happens...
* Always be sure that you unlock SIRs in the inverse order of
locking them, under penalty of you-know-what.
This is an array of warnings that should make the bravest programmer
cringe. Unfortunately, if you want to modify a system table or even
read system tables that you fear might change out from under you, you
have to lock the SIR. So, how can you do this without fearing a system
failure?
* Try to avoid locking SIRs whenever possible. In MPEX, I used to
lock the FMAVT SIR and the FILE SIR (the two SIRs you should lock
when doing certain operations on files -- always lock them in that
order) when I modify a file's file label. Recently, I've changed
it so that I wouldn't modify the file label without first FOPENing
the file exclusively -- that way, I'm guaranteed that no-one else
(except possibly :STORE/:RESTORE) is dabbling with the file, and I
no longer need to lock the SIR. This avoids risk of deadlocks and
system failures.
* When you need to get a SIR, release it as soon as possible. Every
statement between a GETSIR and a RELSIR is just one more
opportunity for a disastrous (system-crashing) abort to occur.
* Write three procedures, SIR'GET, SIR'RELEASE, and SIR'RELEASE'ALL.
- SIR'GET should take a SIR number, turn off break (see above),
lock the SIR, and save the SIR number and the GETSIR result in a
global array. This global array should be shared between these
three procedures, and should actually be treated as a stack,
with new entries being added to the end, and old entries being
deleted (by SIR'RELEASE) in a last-in, first-out fashion.
- SIR'RELEASE should take a SIR number, make sure that it's the
most recently gotten SIR, get the GETSIR result from the
last-added entry in the global array, and call RELSIR.
- SIR'RELEASE'ALL shouldn't take any parameters, but should only
release all the SIRs mentioned in the global array maintained by
SIR'GET and SIR'RELEASE. It should be called by any procedure
that wants to terminate or abort in any fashion. For instance,
the DSEGREAD procedure that I talked about above does a MFDS,
checking its parameters for validity first, and aborting if the
parameters are invalid (to avoid a system failure). If you ever
call it while you have a SIR locked, put a call to
SIR'RELEASE'ALL into it right before it aborts. That way, if you
have a bug and DSEGREAD aborts, it'll release all the SIRs you
have locked before aborting, thus preventing a system failure.
I use these procedures exclusively, almost never calling GETSIR
and RELSIR outside them, and my MPEX (or any of VESOFT's other
privileged mode-using programs) has never crashed the system in
production by aborting while holding a SIR (in fact, they've never
crashed a production system, period). Sometimes, we get a letter
containing a PSCREEN of an abort caused by an MPEX bug (yes, even
MPEX has bugs) with the message "DSEGREAD passed bad parameter,
SIRs released", and I feel kind of proud -- sure, MPEX may have a
bug, but even though it was trying to read a non-existent segment
while it had a SIR locked, it just aborted with a nice message
instead of crashing the system.
The number one cause of user program-caused system failures is not
carelessness, but laziness -- the user didn't spend the time to
develop some simple utility procedures that would prevent minor bugs
or typos from causing system failures.
A final aside on the topic of SIRs -- sometimes (fortunately, very
rarely) you may want to ensure that NOBODY else on the system is doing
anything. Essentially, you want to turn off interrupts to make sure
that until you turn them back on, you and you alone will use the CPU.
This is kind of a "super-SIR" -- you don't just lock some table, you
lock the entire system.
I don't like this, and I've never felt the need to do this. For one,
you can crash the system this way easy as pie -- from what I
understand, any interrupt, including one caused by a request on your
part to access a data segment that isn't currently in memory would
cause a system failure. Furthermore, this isn't as good as a SIR in at
least one way -- when you lock, say, the JMAT SIR, you can be sure
that you will wait until anybody who locked the SIR to modify the JMAT
releases it; that way, when you get it, you know that nobody else is
dabbling with the table. This is not so for turning off interrupts --
the process which is modifying the table you want to modify may have
just been swapped out, and the table might be in a temporarily
incosistent state. Finally, when you turn off interrupts, you also
turn off clock interrupts, and the system clock is essentially not
running until you turn interrupts back on.
As I said, I personally have never needed to do this, and don't
foresee myself doing it in the future. However, if you feel up to it,
there are two machine instructions that do this -- PSDB
(PSeudo-DisaBle) and PSEB (PSeudo-EnaBle).
PSDB disables process dispatching, thus assuring that you'll be the
only one to run until you do a PSEB. Unfortunately, this may (or may
not -- I'm not sure) mean that the system will crash if you try to
access a data segment that is not in memory, or do any other kind of
disc I/O. This means that about the only thing you can safely do
between a PSDB and PSEB is to access system tables that you know are
always memory-resident (e.g. CST, DST, PCB, etc.).
If you have any luck doing this, let me know -- I'd like to see how
this should really be done.
SYSTEM TABLES AND SPL PROGRAMMING STYLE
Everybody and his brother have their own opinions about programming.
The last thing that I want to do is to enter this controversial and
generally thankless field. However, in my experience with using system
tables in SPL, I discovered certain useful programming guidelines that
I want to mention in passing.
The major problem in working effectively with system tables is that
virtually nobody can remember just what the 5th bit of the 53rd word
of an entry in data segment number 22 contains. Furthermore, it is
rather cumbersome to always flip through the System Tables manual
whenever you're writing or reading programs. Comments help some for
program readability, but you'd still have to look at the manual while
writing, and you also stand the risk of incorrect comments.
What one really wants is record structures (like in PASCAL). You
declare a data type PCB'ENTRY'TYPE, declare a data structure of this
type called PCB'ENTRY, read the entry into PCB'ENTRY, and then just
refer to the PCB entry fields as PCB'ENTRY.FATHER'PIN,
PCB'ENTRY.PRIORITY, PCB'ENTRY.CS'QUEUE, etc. If you have another PCB
entry you want to simultaneously work on, just put it into another
data structure -- ANOTHER'PCB'ENTRY -- and use
ANOTHER'PCB'ENTRY.FATHER'PIN, etc.
Unfortunately, SPL does not have these kinds of record structures.
However, they can be cleverly emulated.
What I do is that I declare an array called PCB, and I set up a number
of DEFINEs -- PCB'FATHER'PIN, PCB'PRIORITY, PCB'CS'QUEUE, etc. -- that
refer to the appropriate fields of the array. Then, when the array
contains a PCB entry, these DEFINEs can refer (on either the left or
right side of an assignment statement) to the appropriate fields of
the entry stored in the array. Note that these DEFINEs can refer to
individual bits of the array as well as to entire words. For instance,
one such definition of some of the PCB entries might look like:
<< Definitions of some fields of MPE IV PCB entries. >>
EQUATE SIZE'PCB = 16;
INTEGER ARRAY PCB(0:SIZE'PCB-1);
DEFINE PCB'FATHER = PCB( 5).( 0: 8) #,
PCB'PRIORITY = PCB(13).( 8: 8) #,
PCB'CS'QUEUE = PCB(13).( 2: 1) #;
Note that I also have an equate for the PCB size; I'd also have an
equate for the PCB data segment number, the PCB System Internal
Resource (SIR) number (if it had one), equates or defines for all
interesting PCB constants (e.g. the possible values of the "process
type" field), etc. What's more, I'd put this all into one $INCLUDE
file so that it will be kept in one place and will thus be easily
modifiable.
Other tables are a bit more difficult. The JIT (Job Information
Table), for instance, contains not just word (integer) and bit field
values, but also character arrays and double integers. For that, I
equivalence (using the "BYTE/DOUBLE ARRAY xxx(*)=yyy") a byte array
and a double array to JIT, the main array (which is an integer array).
Thus, the file looks something like:
EQUATE SIZE'JIT=61;
INTEGER ARRAY JIT(0:SIZE'JIT-1);
BYTE ARRAY JIT'B(*)=JIT;
DOUBLE ARRAY JIT'D(*)=JIT;
DEFINE JIT'JOB'ID = JIT ( 9) #,
JIT'MAIN'PIN = JIT (10). ( 8: 8) #,
JIT'GROUP'SEC = JIT'D( 7) #, << Word 14 >>
JIT'ACCT'NAME = JIT'B(32) #; << Word 16 >>
A very few tables contain double integers that are not aligned on an
even word boundary (e.g. word 14), but are rather on an odd word
boundary (e.g. word 21), and thus can't easily be accessed as elements
of a double array equivalenced to the main integer array. For them,
you have to equivalence another double array to the 1st (as opposed to
0th, the default) element of the main array, and then reference them
as elements of this array. For example,
EQUATE SIZE'DIR'GROUP=41;
INTEGER ARRAY DIR'GROUP(0:SIZE'DIR'GROUP-1);
BYTE ARRAY DIR'GROUP'B(*)=DIR'GROUP;
DOUBLE ARRAY DIR'GROUP'D(*)=DIR'GROUP;
DOUBLE ARRAY DIR'GROUP'D'1(*)=DIR'GROUP(1);
DEFINE DG'CPU'COUNT = DIR'GROUP'D'1(6) #; << Word 13 >>
Another advantage of this scheme is that it makes it much easier to
change your programs to work on a new release of MPE in which the
format of a table has been changed -- sometimes you only have to
change the $INCLUDE files and recompile. Also, it makes it
surprisingly easy to have the same source code work with two different
MPE versions (e.g. MPE IV and MPE V) -- just declare a compile-time
switch, say, X5 that is ON when you're compiling the program to run on
MPE V and OFF when you're compiling to run on MPE IV. Then, in all
your $INCLUDE files that describe tables that are different in MPE IV
and MPE V, just check this switch and depending on its value, declare
either the MPE IV or MPE V layouts. For more information on
compile-time switches, see the SPL Reference Manual. An example is the
PCB:
$IF X5=OFF << MPE IV >>
EQUATE SIZE'PCB=16;
INTEGER ARRAY PCB(0:SIZE'PCB-1);
...
DEFINE PCB'STACK'DST = PCB( 3).( 1:10) #;
...
$IF X5=ON << MPE V >>
EQUATE SIZE'PCB=21;
INTEGER ARRAY PCB(0:SIZE'PCB-1);
...
DEFINE PCB'STACK'DST = PCB( 3).( 2:14) #;
...
$IF
The major problem of this method is that it requires the entry to be
stored in a certain fixed place (say, the array PCB). If you have two
entries that you want to manipulate at the same time, you're out of
luck; if you have an entire array of entries that you want to look
through, you have to step through the array, moving each entry to the
array PCB before processing the entry. Also, if you want to access
system tables using SPL memory-resident system table support
constructs (see below), you have to move each word of the table into
the array.
One solution to this problem is to have the DEFINEs contain only the
word number of the element in the table. For instance, for the JIT we
might have:
EQUATE SIZE'JIT=61;
DEFINE JIT'JOB'ID = 9 #,
JIT'ACCT'SEC = 12 #,
JIT'ACCT'NAME = 32 #, << Byte field! >>
JIT'ALLOW'1 = 40 #;
Then, to get the job id, we'd read the JIT entry into anywhere we want
to (say, the array MY'JIT'ENTRY), and access it as
MY'JIT'ENTRY(JIT'JOB'ID). This will work not only for fields that are
whole words, but also for bit fields:
DEFINE JIT'MAX'PRI = 10).(0:8 #,
JIT'MAIN'PIN = 10).(8:8 #;
This way, MY'JIT'ENTRY(JIT'MAIN'PIN) would map to MY'JIT'ENTRY(
10).(0:8 ) -- the right value.
Unfortunately, one of the drawbacks of this method is that if you want
to retrieve a byte field or a double field, you can't just specify it
by name -- you have to explicitly retrieve it as
MY'JIT'ENTRY'B(JIT'ACCT'NAME). If you forget and enter
MY'JIT'ENTRY(JIT'ACCT'NAME), you'll get a very wrong result.
Also, this method requires more typing -- you have to enter both the
field designator (e.g. JIT'ACCT'NAME) and the array neme.
Both methods are acceptable approaches, each with its own advantages
and disadvantages. Use whichever you prefer, or even your own, but
keep in mind the following:
* Use DEFINEs and EQUATEs (for field names, entry size, data segment
numbers, SIR numbers, possible field values, etc.). If you use
"magic numbers" (e.g. bits 12:3 of word 35) in your code, it'll
only make it harder to write, read, and maintain.
* Put DEFINEs and EQUATEs into an $INCLUDE file. That way, if you
need to change something (because you made an error in putting it
in in the first place or because it has changed in the new MPE
release), you'll only have to change it in one place.
AND NOW, FOR SOMETHING COMPLETELY DIFFERENT...
Until now, I have been concentrating mostly on describing how system
tables can be accessed. To do this, I've introduced some important
tables like the PCB and JMAT, but have skimped on description of what
other system tables may exist and what they may contain.
Unfortunately, the System Tables Manual does not really explain
anything except table formats. It'll tell you what the ODD looks like,
but won't tell you thing one about how it fits into the overall system
structure and what you need to know to access it. Once I've whetted
your appetite by telling you how to access system tables, I feel
obligated to present my concept of how all these tables fit together
and where you should go to get a particular piece of data.
As I said before, system tables are just the places where the
operating system "keeps its stuff" -- where it stores information on
all the objects it must manage. A convenient way to look at system
tables is in terms of what objects they describe.
Don't feel nervous if you don't understand the purpose or even the
contents of some of the tables I mention below. Some of them are
really arcane, and may not be worth much to you anyway. If you're just
starting to learn the innards of the system, you might want to skip
reading about everything except the simpler job-, process-, and
file-oriented system tables.
JOB-ORIENTED SYSTEM TABLES
Almost all the work done on an HP3000 is done on behalf of a job
submitted by the user. Note that whenever I say "job", I mean either a
batch job or on-line session, since they are rather similar to the
system. From the system's point of view, a job is a collection of
processes. The Command Interpreter (CI) of a job is one such process.
Any processes you create, either using the :RUN command or using the
CREATE or CREATEPROCESS intrinsics, also belong to that job.
Every job has an entry for it in the Job MAster Table (JMAT). This is
a very simple table; it has a header entry, which contains system
global information (like the number of jobs/sessions, the job/session
limit, etc.), and one entry for each job on the system. Each of these
entries contains some (but not all) information about the job -- its
job name, user, account, group, terminal number (for sessions), the
Process Identification Number of its main process, and so on. A
:SHOWJOB takes virtually all of the information it prints from the
JMAT -- just think of the JMAT as a :SHOWJOB stored in a
machine-readable form. Since there's only one JMAT, it occupies a data
segment with a fixed data segment number. If you want to know the data
segment number or the page of the System Tables Manual on which the
JMAT is described, just look at Appendix 2 of this paper.
A job's JMAT entry, however, does not contain all the information
about the job. Much of the remaining information -- CPU time, the
capabilities of the user running the job, :ALLOW mask, etc. -- is kept
in a table called the Job Information Table (JIT). There's one JIT for
each session, and it's pointed to by the PXGLOB area (which we
mentioned earlier) of the stack of each process that belongs to the
job. To get to your JIT, you'd first find its data segment number from
your PXGLOB, and then use an MFDS to read it. Just as the :SHOWJOB
command reads the JMAT, the WHO intrinsic looks at your JIT; all the
information it gives you -- capabilities, user, account, group, home
group, local attributes, etc. -- all come from the JIT. The JIT is
even easier to work with than the JMAT, being just a big record
structure with a lot of fields (each of which is at least briefly
mentioned in the System Tables Manual). Note that there's some
duplication of data (user name, account name, group name, etc.)
between the JMAT and the JIT. HP does some damnedest things.
If you've been paying particularly close attention, you'll find that I
lied. A job is more than just a collection of processes -- there are
also some job-local entities, such as :FILE equations and temporary
files, that have to be maintained for each job. These are kept in a
Job Directory Table (JDT), of which there is one per session (also
pointed to by the PXGLOBs of the processes belonging to that session).
This table actually contains a header and five sub-tables: one for the
job's temporary files, one for the :FILE equations, one for the JCWs,
one for the job-local data segments, and one for the :CLINE (a
datacomm command) equations. The header contains the pointer to each
of the sub-tables, and each sub-table is in turn is a collection of
variable-length entries. Thus, to find a particular :FILE equation,
you'd get the JDT's data segment number from your PXGLOB, get the
JDT-relative index of the :FILE equation table from the JDT header,
and then go through the :FILE equation table entries until you find
one with the name you're looking for. Not the most entertaining job in
the world, but not too difficult, either.
Finally, for completeness' sake I must mention two other, relatively
unimportant, tables -- the Job CUtoff Table (JCUT) and the Job Process
CouNt Table (JPCNT). The JCUT contains information used for aborting
all jobs that have a CPU time limit. I haven't the foggiest notion of
what the JPCNT is actually useful for, except for a little-used (if at
all) device called Job SIRs. This is one table you can afford to
ignore.
Thus, to summarize, the structure of those system tables that pertain
to jobs is roughly like the following:
JMAT (a single data segment), which contains for each job
The job number, job name, user, account, group, terminal
number, etc.
The Process Identification Number (PIN) of the job's main
(CI) process
One PXGLOB per process belonging to the job, which points to
One JIT per job, which contains
The job number, job name, user, account, group, home group,
CPU time, :ALLOW mask, etc.
One JDT per job, which contains information on the job's
Temporary files (JTFD)
:FILE equations (JFEQ)
Data segments (JDSD)
JCWs (JJCW)
:CLINE equations (JLEQ)
The JMAT entry referring to the job
JCUT (stand-alone table), which contains
Information on all jobs that have a CPU time limit
JPCNT (stand-alone table), which contains
A bunch of bits the reason for which I could never fathom
PROCESS-ORIENTED SYSTEM TABLES
Rather more important than the concept of a job is the concept of a
process.
A process is a single instance of a program running. It has its own
data space and code (the code may be shared among several processes
that are running the same program). All work done on the system is
done on behalf of a process, some of which -- the user processes --
belong to jobs, and others -- the system processes -- do not.
The master table that describes all processes is the Process Control
Block table (PCB). It, like the JMAT, contains a header entry and one
entry for each process. The process entry contains useful information
like a process' priority, the data segment number of its stack, its
family relationships (father, son, brother), wait flags (is it waiting
for a SIR, a RIN, etc.), and more. A process' PIN (Process
Identification Number) is nothing more than the number of the process'
entry in the table. If you have a PIN, you can just multiply it by the
PCB entry size, and do an MFDS of the entry from that location in the
PCB (which has a fixed data segment number).
Also like the JMAT, the PCB does not contain all the information worth
knowing about a particular process. Much very useful information --
what files are open by the process, what its register values are, what
traps it has enabled, where its JIT and JDT are, and so on -- is kept
in the so-called PCB eXtension (PCBX), which is stored in the
DL-negative area of the process' stack. The PCBX consists of
* The PXGLOB, which is mostly the same for all processes in a job
and contains global information like the job's JIT and JDT data
segment numbers, the device on which the job is running, the index
of the process' JMAT entry, etc.
* The PXFIXED, which contains a variety of useful data, like the
values of the process' DB and S registers (they have to be kept
somewhere when the process gets swapped out), the addresses of
whatever control-Y and other trap routines are set, the job number
of the job the process is running, and so on. Like the PXGLOB,
this is just one big record structure.
* The PXFILE, which describes the files the process has open. Its
structure is so complicated that it deserves a separate paper
(which may or may not be forthcoming). Fortunately, the System
Tables Manual is somewhat more lucid than usual with regard to the
structure of the PXFILE (which isn't saying much), and you might
be able to figure things out by reading it. But don't bet on it.
This is about all there is to the structure of the process tables.
Note the recurring concept of a master table that contains some
information on all processes or jobs, and a table for each process or
job that contains more detailed information about the particular
process or job. You'll see this again when we get to logical devices
and the I/O tables.
Finally, two other, less useful, tables. The Process-Job cross
REFerence table (PJXREF), whose format IS NOT GIVEN IN THE SYSTEM
TABLES MANUAL, contains, for every process, its job number. The job
number of process PIN is in the PINth word of this table. The
Process-Process COMmunication table (PPCOM) contains information on
any mail the process might have in its mailbox.
Thus, to summarize, the process table structure looks like:
PCB (a single data segment), which contains for each process
The process' priority, wait flags, etc.
The PINs of the process' father, son, and brother
The data segment number of the process' stack
PCBX (stored in the process' stack's DL-negative area), containing
PXGLOB, which contains
The data segment of the process' JIT and JDT
The index of the process' JMAT entry
Some other information (like the process' terminal)
PXFIXED, which contains
The job number, saved registers, trap addresses, etc.
PXFILE, which contains
Information on all the files used by the process
PPCOM (a single data segment), which contains for each process
Information on any mail message it might have in its mailbox
PJXREF (a single data segment), which contains for each process
Its job number; the job # of process PIN is stored in the
PINth word of the table
DEVICE-ORIENTED SYSTEM TABLES
Many system tables describe devices -- discs, terminals, etc. Of all
the parts of MPE, the I/O system is probably the most intricate and
most confusing portion.
Devices are referred to by their Logical DEVice numbers (LDEVs).
Although this means there's such a thing as a physical device, us
software people don't care about it. In fact, I'll use the terms
device, logical device, and LDEV interchangeably (since everybody else
does, anyway).
Every device has an entry in the Logical Device Table (LDT) and the
Logical-Physical Device Table (LPDT), both of which are single data
segments with fixed data segment numbers. The LDT contains the device
type, record with, main pin of owning process (for terminals and
tapes), etc. Note that the second half of the LDT's data segment
contains the so-called LDT eXtension (LDTX), which contains some more
useful information. Like the LDT, the LDTX has one entry per LDEV. The
start address of the LDTX can be deduced from finding out the number
of configuring LDT entries from the LDT header entry and then
calculating the index of the first word that isn't used by the LDT.
The LPDT contains some other stuff, like the device subtype, state,
some more flags, and the address (SYSGLOB-relative!) of the device's
Device Information Table (DIT).
The DIT contains various information about the device that is highly
specific to the particular device type. The System Tables Manual has
50-odd pages describing the various DIT formats. It is not very
entertaining reading, and will never compete with a well-written, say,
telephone book.
All device classes in the system are listed, along with the LDEVs they
represent, in the Device Class Table (DCT), which contains one
variable-length entry per device class. To find out, say, what devices
the class TAPE contains, just traverse the table until you find an
entry with a name of TAPE. Like the LDT, the DCT shares its data
segment with another (less useful) table called the Terminal
Descriptor Table (TDT).
All the I/Os that are currently pending in the system have entries in
the I/O Queue (IOQ, for non-disc devices) and Disc Request Queue (DRQ,
for disc devices) tables. The last thing I want to do is expend more
energy describing these truly complicated tables that most people will
never use anyway.
Finally, some more useless tables: the Interrupt Linkage Table (ILT),
Driver Linkage Table (DLT), System BUFfers (SBUF), Terminal BUFfers
(TBUF), and the Interrupt Control Stack (ICS). Definitely not your
bread-and-butter programming stuff. If you don't know about them,
you're none the worse for it.
Finally, a recap of the I/O System tables:
LDT (a single data segment), which contains for each device
Device type, record size, main owner pin (for non-discs), etc.
LPDT (a single data segment), which contains for each device
Device sub-type, device status, various other flags, etc.
The address of the device's DIT
DCT (a single data segment), which contains for each device class
The LDEVs that are in the class
The type of class (discs, serial I/O devices, etc.)
DIT (a chunk of memory for each LDEV, pointed to by a
SYSGLOB-relative address), which contains a lot of
device-dependent data
IOQ, DRQ, SBUF, TBUF, DLT, ILT, ICS, LDTX, and TDT,
which you'll have to look up in the System Tables Manual
if you really want to be a big hit at cocktail parties
FILE-ORIENTED SYSTEM TABLES
The entity that everybody works most with is the file.
Every file in the system has a File LABel (FLAB) kept in a sector on
disc. Although this is not a data segment, it can properly be called a
system table, since it contains system-maintained data. The file label
is a veritable treasure-trove of information, containing everything
you'd ever want to know about a closed file. All modes of :LISTF get
their information straight from the file label.
Getting to a file's file label, however, isn't very easy. The disc
address of a file label may be stored:
* for permanent files, in the system directory
* for job temporary files, in the job's JDT
* for spool files, in the Output Device Directory (ODD) or Input
Device Directory (IDD)
* for files that were FOPENed as new files and not yet FCLOSEd, in
the File Control Block (FCB) belonging to that file
For open files, the matter is somewhat more complicated. In addition
to the permanent, relatively unchanging information that is kept in
the FLAB for a closed file, the system must also keep track of an open
file's current record pointer, current block number, buffers, etc. As
I said before, the topic of open file tables is such a complex one
that it deserves a paper of its own. All I'll do here is give a brief
structural description:
FLAB (stored on disc, one per file) contains
The file name, code, size, extent map, etc. -- everything a
:LISTF could give you
The system directory contains entries that point to FLABs of
permanent files
Each job's JDT contains entries that point to FLABs of job temporary
files
The spool file directories IDD and ODD contain entries that point to
FLABs of spool files
Each process' Active File Table (AFT), stored in the process' PXFILE
area, briefly describes all the files opened by the process (a
file number is an index into this table) and points to each file's
LACB and PACB
For each open of a file, a Physical Access Control Block (PACB)
describes the current state of the file -- the current record
number, the buffers used for buffered file access, the number of
physical and logical I/Os that have occurred, and so on. For files
opened MULTI, or GMULTI, there's only one such PACB for all
openers; for all other open files, there's one PACB per opener.
For each open of a file that is being accessed using MULTI or GMULTI
access, there's one LACB per opener; for files not opened using
MULTI or GMULTI there are no LACBs
For each opened file, there's an FCB, which contains information
common to all readers of the file (mostly a rehash of the file
label, kept in memory for faster access). The FCB is pointed to
both by all the file's PACBs and by the file's file label.
FMAVT (File Multi-Access Vector Table) contains information on all
files opened with MULTI or GMULTI access
SEGMENT-ORIENTED SYSTEM TABLES
Segments are broken up into two very distinct categories: data
segments and code segments.
Data segments are described in the Data Segment Table (DST); there's
one entry for each data segment, and it contains information like the
data segment length, memory address, flags, etc. That's all there's to
it, a welcome change from the complicated structures in place for
files, devices, and even jobs and processes.
No such luck with code segments, though. The Code Segment Table (CST)
contains only the entries -- also listing the segments' lengths,
addresses, and other flags -- referring to all currently loaded SL
segments and the program segments of the process that is currently
running.
Say that you want to find out about the code segments of a process
given its PIN. What you want to do is to get to the entries of the CST
eXtension (CSTX) table -- which is stored in a single data segment --
that correspond to the process' code segments.
Your first step would be to get the "CSTX block map index", known as
CSTXEIX, of the process from the process' PCB. Note that this value is
listed in the PCB table layout as BLKINX and PBX.
This is an index into a table called the CSTBLK, also known as the
CSTXMAP. The CSTXEIXth entry in the CSTBLK contains one thing and one
thing only: the address of the first CSTX entry describing the
segments of the process.
However, this is NOT a CSTX-relative address. This is not even a
SYSGLOB-relative address. An absolute address, you say? No.
This address is relative to the base of the DST, an entirely different
data segment altogether! Since you can't very well MFDS CSTX entries
from the DST, you have to subtract the contents of SYSGLOB location
%33 from the value, and then you have the CSTX-relative index. SYSGLOB
%33 (absolute %1033) is the DST-relative address of the first entry of
the CSTX.
Once you're done with all this -- you got the CSTXEIX from the PCB,
you got the DST-relative address from the CSTBLK, and you converted it
to a CSTX-relative-address -- you can now get the CSTX entries. The
first entry, the one pointed to by the address you worked so hard to
get, tells you how many code segments the process' has and how many
users are sharing it. Subsequent entries describe each of the
program's segments.
In summary, the segment-oriented tables look like this:
DST, one entry per data segment containing
The length of the data segment
The memory address of the start of the data segment
Miscellaneous flags
CST, one entry per SL code segment and each code segment of the
currently-executing program containing
The length of the code segment
The memory address of the start of the code segment
Miscellaneous flags
CSTX, one entry per loaded program (not process! all processes
running the same program share code segments) containing
The number of segments in the program
The number of processes running the program
Also in the CSTX, one entry per segment of each loaded program,
formatted like a CST entry
CSTBLK (also known as CSTXMAP), one entry per program containing
A DST(!)-relative pointer to the first CSTX entry belonging
to the program
MISCELLANEOUS SYSTEM TABLES -- SPOOLING
The information on all spool files in the system is kept in the Input
Device Directory (IDD) for input spool files and the Output Device
Directory (ODD) for output spool files. Both of these are independent
data segments, and the formats of all their entries are identical. XDD
is a term used to generically talk about either the IDD or the ODD; an
XDD entry is an entry that could be from the IDD or the ODD.
An XDD has three kinds of entries:
* One header entry, describing the table -- it contains things like
the size of the table and the next input/output spool file number
to be allocated.
* One device header entry for each device in the system. Each entry
contains the device number, device outfence (if any), and the head
and tail addresses of the linked list of spool file entries
belonging to that device. The 0th device header entry contains the
head and tail addresses of the linked list of all spool file
entries belonging not to a device, but to a device class.
* One spool file entry for each existing spool file. Each entry
contains the name of the spool file, the user, account, and job
name that it belongs to, the disc address of its file label, etc.
SPOOK's >SHOW command (especially >SHOW;@) essentially print the
contents of these entries.
This is a comparatively easy table to navigate. MPEX's spool-file
manipulation routines -- %PURGE:SPOOL, %LISTF:SPOOL, etc. --
essentially traverse the ODD, picking up spool file file labels as
they go along. Note that the addresses in the linked list are
table-relative addresses of the start of the next spool file entry; 0
indicates no next entry.
MISCELLANEOUS SYSTEM TABLES -- OTHERS
There are several other relatively useful system tables.
The LST (Loader Segment Table) contains information on all loaded
program files and SL files. MPEX's %LISTF,ACCESS goes through this
table to find out who may have a particular file loaded.
The SIR (System Internal Resource) table contains information on all
SIRs that are currently locked. If you see the system hanging up and
you're afraid that it's because someone isn't releasing a SIR or
you're getting a SIR deadlock, you can go into privileged mode :DEBUG
and have a look at this table.
The RIN (Resource Identification Number) table contains information on
all RINs that are currently locked. MPEX's %LISTF,ACCESS looks at this
to find out who is locking a file or waiting to lock the file.
The RIT (Reply Information Table) contains information on all
outstanding replies.
There are other tables, having to do with logging, private volumes,
message files, :WELCOME messages, you name it. I won't talk further
about them simply because of space limitations. Also, most of the
stand-alone tables are relatively simply structured, and you can find
out how to get to them by reading the System Tables Manual and
conducting some experiments with privileged mode :DEBUG.
Remember that there is a brief description of every (well, almost
every) table in the system and a "pointer" to its entry in the System
Tables Manual in Appendix 2 of this paper.
CONCLUSION
System tables are not the incomprehensible monsters that they may seem
to be at first glance. Once you master the various techniques
described in this paper and get a reasonable familiarity with the
actual contents of the tables, you should, with relatively little
effort and worry, be able to access system tables just as easily as
you would, say, a file or a database.
Good luck and HAPPY HACKING.
ACKNOWLEDGEMENTS
I would like to thank the following people for reviewing and making
many useful comments on this paper:
Steve Cooper, Allegro Consultants
Robert Green, ROBELLE Consulting Ltd.
David Greer, ROBELLE Consulting Ltd.
Jack Howard, Consultant, Los Angeles
Stan Sieler, Allegro Consultants
Jim Squires, HP Fullerton
Vladimir Volokh, VESOFT, Inc.
and of course everybody -- too numerous to mention -- who taught me
all this stuff. May your programs never fail and your systems never
crash.
APPENDIX 1 -- PROGRAM 1
<< Don't forget to :PREP me with ;CAP=PM! >>
<< Works on MPE IV and MPE V. >>
$CONTROL MAIN=PROGRAM'1, NOSOURCE, USLINIT
<< This program:
Prints out word 0 of the user's capability matrix;
Prints out the job or session's job/session number;
Gives the user SM capability and does a LISTUSER
before (which should fail) and after.
>>
BEGIN
INTRINSIC
ASCII,
COMMAND,
GETPRIVMODE,
GETUSERMODE,
PRINT;
INTEGER POINTER
DL;
INTEGER
CAPS,
DUMMY,
ERROR,
JS'NUMBER,
LENGTH;
ARRAY
BUFFER'L(0:127);
BYTE ARRAY
BUFFER'(*)=BUFFER'L;
DEFINE
INDEX'PXGLOB = DL(-1) #,
INDEX'PXFIXED = DL(-2) #;
<< Get the DL pointer >>
PUSH (DL);
@DL:=TOS;
<< Get and print word 0 of the capability matrix >>
GETPRIVMODE;
CAPS:=DL(-INDEX'PXGLOB+2);
GETUSERMODE;
MOVE BUFFER':="Capability word 0 = %";
PRINT (BUFFER'L, -21, %320);
ASCII (CAPS, 8, BUFFER');
PRINT (BUFFER'L, -6, 0);
<< Get and print the session number >>
GETPRIVMODE;
JS'NUMBER:=DL(-INDEX'PXFIXED+19);
GETUSERMODE;
IF JS'NUMBER.(0:2)=1 THEN
<< JS'NUMBER.(0:2) indicates session (1) or job (2) >>
MOVE BUFFER':="#S"
ELSE
MOVE BUFFER':="#J";
PRINT (BUFFER'L, -2, %320);
LENGTH:=ASCII (JS'NUMBER.(2:14), 10, BUFFER');
PRINT (BUFFER'L, -LENGTH, 0);
<< First, show that LISTUSER MANAGER.SYS fails without SM >>
MOVE BUFFER':="Trying LISTUSER MANAGER.SYS before getting SM";
PRINT (BUFFER'L, -45, 0);
MOVE BUFFER':=("LISTUSER MANAGER.SYS", %15);
COMMAND (BUFFER', ERROR, DUMMY);
IF ERROR<>0 THEN
BEGIN
MOVE BUFFER':="CI Error # ";
PRINT (BUFFER'L, -11, %320);
LENGTH:=ASCII (ERROR, 10, BUFFER');
PRINT (BUFFER'L, -LENGTH, 0);
END;
<< Now, get SM capability >>
GETPRIVMODE;
DL(-INDEX'PXGLOB+2).(0:1) << SM bit >> := 1;
GETUSERMODE;
<< Now, show that LISTUSER MANAGER.SYS succeeds >>
MOVE BUFFER':="Trying LISTUSER MANAGER.SYS after getting SM";
PRINT (BUFFER'L, -44, 0);
MOVE BUFFER':=("LISTUSER MANAGER.SYS", %15);
COMMAND (BUFFER', ERROR, DUMMY);
IF ERROR<>0 THEN
BEGIN
MOVE BUFFER':="CI Error # ";
PRINT (BUFFER'L, -11, %320);
LENGTH:=ASCII (ERROR, 10, BUFFER');
PRINT (BUFFER'L, -LENGTH, 0);
END;
END.
APPENDIX 1 -- PROGRAM 2
<< Don't forget to :PREP me with ;CAP=PM! >>
<< Works on MPE IV; changes needed for MPE V are indicated. >>
<< This program assumes the existence of the DSEGREAD and MYSTACK
procedures described in the text. >>
$CONTROL MAIN=PROGRAM'2, NOSOURCE, USLINIT
BEGIN
<< This program:
Prints out the job/session name of the current job/session
(or blanks if no job/session name.)
>>
INTRINSIC
PRINT;
EQUATE
SIZE'PXGLOB = 8; << 12 for MPE V >>
DEFINE
PXGLOB'JIT'DSEG = PXGLOB(6).(6:10) #; << JIT data segment # >>
<< PXGLOB(11) for MPE V >>
ARRAY
PXGLOB(0:SIZE'PXGLOB-1); << Array for holding the PXGLOB data >>
EQUATE
OFFSET'PXGLOB = 0; << Offset of the PXGLOB in the stack dseg >>
EQUATE
SIZE'JIT = 61; << 67 for MPE V >>
DEFINE
JIT'JS'NAME = JIT(44) #; << MPE IV or MPE V >>
ARRAY
DSEGREAD (MYSTACK, OFFSET'PXGLOB, PXGLOB, SIZE'PXGLOB);
DSEGREAD (PXGLOB'JIT'DSEG, 0, JIT, SIZE'JIT);
PRINT (JIT'JS'NAME, -8, 0);
END.
APPENDIX 2 -- SUMMARY OF SYSTEM TABLES
System tables are listed alphabetically by their initials. The chapter
numbers of the tables in the System Tables Manual are given. Page
numbers are not given because they may vary from release to release of
each manual -- they shouldn't be too hard to find in any case.
All tables are stored in memory (not on disc) unless otherwise
specified.
Table Chapter Table describes or contains
----- ------- ---------------------------
AFT 6 [3] One/process; part of PXFILE
ASSOC 15 All :ASSOCIATE commands
BKPNT 17 :DEBUG breakpoints
CBT 6 [3] Contains file system control blocks
CILOG None :(CMD) USER.ACCT logons
CST 2 * Loaded SL and current program's code segments
CSTAB 23 [2] Contains communications system info
CSTBLK 2 * Contains pointers into the CSTX
CSTX 2 All code segments in the system
DCT 13 Device classes
DIT 13 [3] One/device; contains device-specific info
DLT 13 Device drivers
DRQ 13 * Queued disc I/O request
DST 2 * Data segments
FCB 6 [3] One/file opener; file system info
FLAB 6 [3] One/file, stored on disc; file parameters
FMAVT 6 Files opened with MULTI/GMULTI access
ICS 13 Contains stack of interrupt-handling process
IDD 14 Input spool files
ILT 13 Contains interrupt info
IOQ 13 * Queued I/O requests
JCUT 8 * Jobs that have a CPU limit
JDSD 8 [3] One/job; job data segment info, part of JDT
JDT 8 [3] One/job; job's :FILE eqns, temp files, etc.
JFEQ 8 [3] One/job; job :FILE equation info, part of JDT
JIT 8 [3] One/job; detailed job info
JJCW 8 [3] One/job; job JCW info, part of JDT
JLEQ 8 [3] One/job; job :CLINE equation info, part of JDT
JMAT 8 Jobs
JPCNT 8 * Contains arcane info about jobs
JTFD 8 [3] One/job; job temp file info, part of JDT
LACB 6 [3] One/MULTI or GMULTI file opener; file system info
LDT 13 LDEVs
LDTX 13 LDEVs
LIDTAB 17 Logging identifiers
LOGBUFF 17 Contains logging buffers
LOGTAB 17 Logging
LPDT 13 * LDEVs; points to DITs
LST 11 All loaded program and SL files
MEASINFO 17 * Measurement info
MVTAB 12 Mounted private volumes
ODD 14 Output spool files
PACB 6 [3] One/file opener; file system info
PCB 7 * Processes
PCBX 7 [3] One/process; contains PXGLOB, PXFIXED, PXFILE
PJXREF None Maps PINs into job numbers
PPCOM 7 MAIL messages that have been sent but not gotten
PVUSER 12 Users of private volumes
PXFILE 6 [3] One/process; contains info on process' files
PXFIXED 7 [3] One/process; contains miscellaneous process info
PXGLOB 7 [3] One/process; points to JIT and JDT
RIN 5 RINs (including file locks)
RIT 15 Outstanding :REPLYs
SBUF 13 * Contains buffers used for some system I/O's
SIR 5 * Locked SIRs
SRT 2 * Special memory management requests
SWAPTAB 2 * Contains segment swapping info
SYSGLOB 1 Contains miscellaneous system info
SYSJDT 8 Contains the JDT used by system processes
SYSJIT 8 Contains the JIT used by system processes
TBUF 13 * Contains buffers used for terminal I/O's
TDT 13 [1] Special configurations for terminals
TRL 17 * Requests for system timer (PAUSEs, timeouts, etc.)
UCRQ 8 All requests to the process controller process
VDD 17 Mounted labelled tapes
VDSMTAB 3 * Virtual memory on discs
VTAB 3 Discs
XDD 14 Generic name for either the IDD or ODD
* - Indicates a system table that is accessible using the SPL "INTEGER
POINTER table = number;" construct. For the system table number
(not necessarily equal to the data segment number), see the layout
of SYSGLOB in chapter 1. If SYSGLOB cell N is indicated as
containing the base of a table, then N is the table's number.
[1] - Indicates a table that exists in MPE V but not in MPE IV.
[2] - Indicates a table documented in the MPE V manual but not in the
MPE IV manual.
[3] - Indicates that there is more than one of these tables, one for
each one of a number of objects. For instance, a JIT, of which
there's one per job, and an FLAB, of which there's one per file.
All tables not so marked are unique.