THE SECRETS OF SYSTEM TABLES... REVEALED! by Eugene Volokh, VESOFT Presented at 1985 HPIUG Conference, Washington, DC, USA Published by SUPERGROUP Magazine, Nov 1984/Apr 1985. Published in "Thoughts & Discourses on HP3000 Software", 3rd ed. The System Table is an Ornery Beast That defies investigation Unless one first acquaints oneself With the proper incantations. Long must one labor o'er Deep-hidden books of arcane lore Until one learns the secrets of EXCHANGEDB, MFDS, and more. And formats one must understand What's in word one, what's in word two What lurks inside the fourteenth bit And what the sixteenth SIR can do. A systems programmer's what Wilfred was, With stolid heart and trusty blade, And system tables he did read, And many useful programs made. Oh, listen to his fearsome tale Of magics dark and dragons fierce, And catch a glimpse of mysteries Not often told upon this earth. -- "The Saga of Wilfred," Norse, ca. 11th century AD. ABSTRACT MPE's System Tables contain a large variety of data that can be very useful for any task from performance optimization to writing system management utilities to improving system security and adding power to application programs. Unfortunately, nowhere is it clearly explained (or for that matter, explained any way at all) how one can access all these system tables. The goal of this paper is to shown how one can access data segments, file labels, absolute memory locations, disc-resident system tables -- in general, all forms of system tables -- so that armed with the information gleaned from here and a System Tables Manual, a programmer can sit down and start writing programs that manipulate data segments. INTRODUCTION Just like any other program, the operating system must keep track of its current state; information on all the things known to it -- files, jobs, programs, everything -- must be kept somewhere. A "System Table" is just a name for a data structure containing such data. System tables may be stored on disc, in memory as a fixed array of memory locations (e.g. locations %1000 to %1377 of bank 0), in memory as a data segment or a portion thereof, in the stack of some system process or even a user process -- any data structure maintained by the operating system can be legitimately called a system table. There are two problems with using system tables, which account for the fact that few people know how to use them safely -- one is that the format of the tables is little-known, and is usually documented only in the System Tables Manual (not the most readable document known to man) or, worse, only in the source code of the procedures that actually maintain the data structure. Even when the format is documented (e.g. "word 7 contains the flumbrug framastat" is a typical example), various important details (just what is a flumbrug framastat) are omitted. Equally bad is the other major problem -- even if one knows the format of a system table, nowhere does it clearly describe how to access it. This paper will try to address both these problems. It will give instructions on how to access various system tables, but will also explain what the various system tables are and how they are inter-related. This, coupled with the precise formats given in the System Tables Manual, should allow the interested reader to learn to manipulate much useful system information. If you don't have the System Tables Manual, you can order it from HP as part number 32002-90003 (MPE IV) or 32033-90010 (MPE V). Note that although all the examples in this paper are given in SPL, there is no reason why you can't write system table-accessing programs in PASCAL, FORTRAN, or even COBOL. All that is necessary is to write simple SPL routines for performing those operations (such as MFDS or TURNOFFTRAPS) that can only be done in SPL, and then put the routines in an RL or an SL. That way, the routines will be callable from your favorite language. System procedures like FLABIO, ATTACHIO, or GETSIR can already be called from other languages. "WAITER, THERE'S A SYSTEM TABLE IN MY STACK!" It might come as some surprise to you that one place the system hides a system table is in your very own stack. Your stack is a "data segment," a contiguous chunk of memory no more than 32K words long, in which addressing is done relative to the "data base" (DB) register (not to be confused with IMAGE databases). If you draw your stack (with addresses growing toward the top), it would look something like this (see Section I of SPL Manual): High memory Z > --------------------------------------------- ^ | unused space | ^ S > --------------------------------------------- ^ | operands of machine instructions | ^ | and local variables of the currently | ^ | executing procedure | ^ Q > --------------------------------------------- ^ | local variables of other procedures | ^ | and stack markers indicating calls from | ^ | one procedure to another | ^ Qi > --------------------------------------------- positive | global variables | addresses DB > --------------------------------------------- | "DB negative area," accessible only by | negative | SPL procedures (like V/3000) -- usable | addresses | for global storage | v DL > --------------------------------------------- v | The Nether Regions, where mortals may | v | not stray and non-privileged accessors | v | are punished with bounds violations | v --------------------------------------------- v Low memory At the very top is the Z register (stands for Zounds?), which is the uppermost boundary of the stack data segment (note that in some manuals, HP draws the stack upside down, with DL on top and Z on bottom -- keep this in mind when comparing this picture with others). Then, below it is the S register, which marks the "top of the stack"; parameters to machine instructons are kept at and around S. Below S is the Q register, which indicates the top of the stack at the time the currently-executing procedure was called; the procedure's local variables are allocated above it, with the procedure's parameters and the stack marker (indicating the place from which the procedure was called) immediately below it. "Qi" is actually not a register, but rather the initial value of the Q register (which changes with every procedure call and every procedure exit). DB is the base register, from which all addressing is done. Word address 6680 is actually "DB+6680"; any pointers that are kept (even if they are pointers to local variables, which are between the Q and S registers) are relative to DB, since DB is the only register that is guaranteed not to change. DL (which - since it, like all registers save for DB itself, is expressed as a DB-relative address - is a negative address) marks the lowest point in the stack that is accessible to user code; it, too, can be moved to dynamically expand or contract the useful DL-DB area, in which SL procedures (like V/3000) can store global data. What interests us is the "nether reaches," the area below DL. Your mother may not have told you this, but there is a whole wealth of information (for many programs I've seen, more interesting than the stuff above DL) called the "PCBX" tucked away there. "PCBX" stands for "PCB eXtension" (the PCB being an important system table that you'll hear more about later), and contains much of the information that the system needs to know about the process -- what files it has open, what session it belongs to, where other tables that describe the session more thoroughly (the JIT and JDT) are, what data segments this process has allocated that must be deleted when it dies, and so on. The PCBX is divided into four portions (see Ch.7 of Sys.Tables manual): - PXGLOB, which lists things like the capabilities of the person who is running the process, the DST indexes (aka data segment numbers, which we'll talk more about later) of the Job Information Table (JIT) and Job Directory Table (JDT) pertaining to the process's session, and so on. - PXFIXED, which describes some other things about the process -- what data segments it has allocated, what session it belongs to, what the values of its registers are (if it is not currently active; if it is currently activate, i.e. being executed by the CPU, the register values are actually stored in the CPU registers), and so on. - PXFILE, which describes the files that the process has opened. - The pointer area, which points to the other portions. Unlike most other pointers, these pointers are DL relative and are implied to be negative; i.e. a value of 364 points to DL-364. They are arranged roughly as follows: DL > --------------------------------------------- DL-1> | DL-relative pointer to PXGLOB | --------------> DL-2> | DL-relative pointer to PXFIXED | ----------> | DL-3> | DL-relative pointer to PXFILE | ------> | | DL-4> | Special count that we don't care about | | | | --------------------------------------------- | | | | | | | | | PXFILE | | | | | | | | | --------------------------------------------- <-----v | | | | | | | PXFIXED | | | | | | | --------------------------------------------- <---------v | | | | | PXGLOB | | | | | --------------------------------------------- <-------------v Now, how does one access these system tables? Well, they are kept in your stack, so they can be accessed using simple SPL DB-relative pointers. For instance, to get word 2 of the PXGLOB (which happens to be, in both MPE IV and MPE V, the zeroth word of the user's capability mask, containing information on whether the user has SM capability, AM, etc. -- all the capabilities save for PM, PH, DS, MR, IA, and BA) you would merely have to do the following: INTEGER POINTER DL; INTEGER CAPS; PUSH (DL); << push the address of DL onto the stack >> @DL:=TOS; << DL now points to the DL register >> GETPRIVMODE; << all DL-negative addressing must be done in priv >> CAPS:=DL( -DL(-1)+ << -DL(-1) is the DL-relative ptr to PXGLOB >> 2); << add 2 to get to the caps word >> GETUSERMODE; Voila! Note that we have to be in privileged mode to access ANY word of the PCBX. After all, if you could do this in user mode, you could change the line "CAPS:=DL(-DL(-1)+2)" to "DL(-DL(-1)+2):=CAPS" and CHANGE the capabilities word instead of just reading it... in fact, a variation of this is exactly what VESOFT's GOD program (which temporarily -- for the duration of the session -- gives the user who ran it all the capabilities and all the ALLOWs) does. Similarly, one can get at all the other portions of the PCBX. For instance, to get the Active File Table (AFT) entry for file number FNUM, one can do the following: INTEGER POINTER DL; INTEGER ARRAY COPY'OF'AFT'ENTRY(0:5); PUSH (DL); @DL:=TOS; GETPRIVMODE; MOVE COPY'OF'AFT'ENTRY:=DL(-6*FNUM-4),(6); GETUSERMODE; Of course, you must first know that the AFT entry for file FNUM is 6 words long and always starts at location DL-6*FNUM-4 (in MPE V only -- this is different from MPE IV). Also note that the AFT is actually known as either the Active File Table or the Available File Table (depending on which chapter of the System Tables Manual you believe). You can look up its format in chapters 6 and 7 of the manual. If you want a brief exercise in using this access method, whip out your trusty System Tables Manual and write a small program to print out your current capability word (in octal) and your current job/session number (HINT: it's in the PXFIXED). While you're at it, also change your current capability word to give yourself SM capability (make sure you do not already have it) and do a "LISTUSER MANAGER.SYS" using the COMMAND intrinsic before and after this change. You don't need to change it back since it is valid for the duration of the process only, and when the process dies, the user is left with the same capabilities as before. The solution to this problem (which we'll call Problem 1) is in Appendix 1. Of course, when you're learning to use these access methods, be sure to run all your test programs during off hours or on a non-production computer. I've crashed my share of computers when I was starting to work with system tables, and you'll probably crash some, too. Fortunately, once you get acquainted enough with the techniques involved, and especially if you use some of the safety devices I'll describe later, your programs will probably be far more reliable. In the meantime, expect some foul-ups. Another useful trick is using privileged mode DEBUG to ensure that your program is working right. Say your program doesn't produce the correct result -- it could either be because you're trying to get the wrong value (e.g. PXGLOB location 3 doesn't really contain what you think it contains) or that you're going for the right value, but your method of getting it is wrong or you're outputting it wrong. To find out what the problem is, you can use DEBUG to look at the value manually -- if it's the expected value, then the bug is in your retrieval or output algorithms; if it's not the expected value, you're trying to retrieve from the wrong place. The way to get at values in PXGLOB using privileged mode DEBUG (you must be a privileged user to do this!) is to type a command of the type "DDL-address,length". Don't forget that DEBUG by default assumes octal input and output; to input a decimal number, prefix it with a "#"; to make output come out in decimal, type "DDL-address,length,I". Thus, to see the value of DL-1 (the PXGLOB pointer), type "DDL-1,1,I", or just "DDL-1,I" since 1 is the default length. To find the value of PXGLOB word 3, you can either first do a "DDL-1,I", and do "DDL-#xxx+3,I" (where xxx is the value that the DDL-1 gave you), or you can say "DDL-'DL-1'+3,I" -- the "'DL-1'" means "the value stored at DL-1". One important note: although the access method I just described works well enough, I favor a different approach that I'll discuss when I get to system tables kept in arbitrary data segments and the MFDS instruction. THERE'S DATA IN THEM THAR DATA SEGMENTS If you peek into your friendly neighborhood System Tables manual (which at last count had 23 chapters), you will find that about half of chapter 7 is dedicated to tables that reside in stacks' DL-negative areas, and you might start to wonder what the remaining 22 1/2 chapters are about. Well, most of the system tables described in the manual are stored in operating system Data Segments, and if you thought that DL-negative tables were tough to access, just wait until you see these. It doesn't make sense to keep data in fixed locations in memory, e.g. putting the table of currently active jobs in locations %10400 through %10777, a process's stack in locations %533000 through %534377, and so on. For one, the very concept of "virtual memory" demands that the operating system be able to swap data out from disk and then bring it back into a different location in memory; furthermore, the HP3000 is still a 16-bit machine (yes, Virginia, there are still 16-bit machines out there, and you're using one of them), and it would have trouble supporting addresses that are more than 16 bits long. Rather, the data is broken up into many "segments". Within each segment, all addresses are relative to the base of the segment; if one segment has to point to another, it would have both the segment number and the offset within the segment. This way, a segment can be conveniently swapped in and out of memory, since the physical address of the segment is kept in only one table that is managed by the swapper; also, since most addresses are segment-internal, we can usually get away with quick 16-bit addressing. Consider for instance your own process stack -- it, too, is a data segment. Addresses within it are relative to the process's DB register (a slight difference from other segments, in which the addresses are relative to the segment's base); the stack contains all your data plus the tables in the DL-negative area that we talked about. In case you should wish to access another process's stack, you can -- all you need is its "data segment number." Similarly, the table of all the currently active jobs (called the "JMAT," for Job MAster Table) is also kept in its own data segment, as is the table describing all currently active processes (the PCB -- Process Control Block). Any such table can be accessed if you only know its data segment number. Incidentally, let me mention a certain documentation inconsistencies pertaining to data segments. What I call the "data segment number" is sometimes called: A "DST index." All existing data segments are described in a system table (which itself is in a data segment) called the Data Segment Table, a k a DST. The number of the data segment is thus the index into that Data Segment Table. A "DST number," a corruption of "DST index." Doesn't make much sense -- Data Segment Table number? A "DST," a further corruption of "DST index." Although this is actually both ambiguous and technically improper, lots of people use it (including yours truly). It's all HP's fault that they didn't call it something decent like "data segment number" in the first place. I shall make a daring attempt to CONSISTENTLY call these critters data segment numbers throughout this document, but don't count on it. ACCESSING DATA SEGMENTS -- ONE APPROACH We have established that most of the data worth knowing is kept in data segments, and that a piece of data can be accessed by specifying the data segment number and the offset within the data segment, without caring about the physical memory address, which anyway will probably change when the segment is swapped in and out. Now, the question becomes: how do you get to it? Well, thought some unknown HP designer when he was faced with making all this up back in '72 when the 3000 was being built, we can already access arbitrary DB-relative addresses in our stacks -- why don't we just provide some way of switching the DB register to point to an arbitrary data segment? So, say that someone wants to get location 3 of data segment 55 -- he'll just have to switch to segment 55, grab location 3 (note that he'll do it just like he'd grab location 3 of his own stack, except that now DB points to segment 55), and switch back to his stack, the value of the desired location in hand. Thus, our program would look something like: << ATTENTION: People who're just looking at the pictures: THIS PROGRAM WON'T WORK! DON'T EVEN TRY IT! Read the text... >> $OPTION PRIVILEGED << PM or no dice! >> BEGIN INTEGER DUMMY, << location DB+0 >> I; << location DB+1 >> << This is a system procedure that switches DB to point to the specified data segment, and returns the number of the data segment to which it now points. If the data segment number is 0, switches to the process's stack. This is NOT the same as the documented privileged procedure SWITCHDB -- THEY CAN NOT BE USED INTERCHANGEABLY! SWITCHDB is only intended for accessing your own data segments that were allocated by your program using GETDSEG. >> INTEGER PROCEDURE EXCHANGEDB (DSEG'NUMBER); VALUE DSEG'NUMBER; INTEGER DSEG'NUMBER; OPTION EXTERNAL; DUMMY:=EXCHANGEDB (55); ASSEMBLE (LOAD DB+3); << push the value at DB+3 onto the stack >> I:=TOS; << pop it from the stack -- I now contains the value of the 3rd location in segment 55 >> DUMMY:=EXCHANGEDB (0); << switch back to the stack >> << now, do whatever we want to do to I >> END. What did we do? We set our DB register to point to data segment 55, we got the value at location DB+3 -- now location 3 of segment 55 -- and moved it into I, and then we switched back to our stack. Note that we did not use the value returned by the first EXCHANGEDB, which is the data segment number of the segment to which DB pointed to before -- namely, our stack -- as the data segment number in the second EXCHANGEDB. That's because if we just EXCHANGEDB to our stack's data segment number, DB will point to the BASE of the stack data segment; however, specifying 0 as the data segment number is a special feature that switches back to the process's stack data segment and makes DB point to the right place. But THIS DOESN'T WORK. What's more, IT FAILS IN A TRULY SPECTACULAR WAY, most likely crashing your system or worse. Why? Because when we do an EXCHANGEDB, ALL THE DB-RELATIVE ADDRESSES NOW POINT INTO THE DATA SEGMENT, including all of our DB-relative variables -- such as I -- and our arrays, either DB-relative (global), or Q-relative (procedure local)! When we move the top of stack into I, we are moving it to DB+1 -- the location in which I is stored -- and DB+1 now points into the data segment, too! In fact, the only way in which the above could be made to work is by making sure that whenever we use any DB-relative addressing while in "split-stack mode" -- i.e. when we've EXCHANGEDBd to another segment -- we mean addressing relative to the data segment. For instance, we could leave the value of location 3 on the stack, EXCHANGEDB back to our stack data segment, and then safely pick the value up into I; or, we could put this in a procedure, and make I a Q-relative variable. Remember, Q-relative and S-relative addressing does not get changed; however, ALL DB-relative addressing does. The bottom line is that, for all practical purposes, we can have access to only one segment at a time -- either our stack, in normal mode, or one alternate data segment in split-stack mode. We can't really work with both, except when we're in split-stack mode and we use S-relative and Q-relative addressing. Remember, though that even this is restricted since most arrays, even Q-relative ones, use DB addressing anyway (because they have a pointer that points to the data, and all these pointers are always DB-relative). So, if for the entirety of our process's life we want to be dabbling with data segment 55, we have no problem; however, if we usually want to work with our stack -- remember, data segment 55 probably does not have room for our temporary variables, arrays, and whatever else is usually kept in our stack -- and only sometimes get at data segment 55, we have a problem. Thus, the rules of the game for split-stack mode operation are: * Always be keenly conscious of when you are in split-stack mode. Try to be in split-stack mode as little as possible (I prefer never to be in split-stack mode except when some operating system procedure that I'm calling -- such as DIRECSCAN, the directory traversal procedure -- requires it). * When in split-stack mode, remember that only: - By-value procedure parameters; - Simple (non-array) procedure-local variables; - Variables explicitly declared to be Q-relative or S-relative (using constructs like "INTEGER XYZZY = S-3;"); - Local arrays that are declared to be Q-relative (e.g. "INTEGER ARRAY X(0:4)=Q;"); refer to data in your stack; all others refer to data in the data segment to which you've switched. A CIVILIZED ALTERNATIVE Fortunately, there is another way of accessing data segments -- two little-known privileged machine instructions called MFDS (Move From Data Segment) MTDS (Move To Data Segment) Their principle of operation is simple -- they move data from the stack to a data segment or from a data segment to the stack. Thus, if you want to read location 3 of data segment 55, you'd issue an MFDS instruction indicating that you want to move 1 word from location 3 of data segment 55 to some buffer in your stack. It's simple, and since you don't do an EXCHANGEDB, you can keep using all of your variables with no problems. Now, let's be a bit more specific: just how does one tell the instruction where to move from and where to move to? Well, the best place where instructions can take input parameters is from the stack -- not just from your stack data segment, but from the topmost few locations on the stack, just below the S register. You can put parameters onto the stack using the SPL construct TOS:=parameter value; and then issue the instruction by entering ASSEMBLE (MFDS 4); or ASSEMBLE (MTDS 4); The "4" in the ASSEMBLE statement is the so-called "stack decrement" -- it indicates how many of the parameters passed to the instruction should be removed from the stack once the instruction is done. Each instruction takes 4 parameters (which we'll talk about soon), and there's no reason to leave any of them laying around on the stack, so we tell the instruction to get rid of all 4 of them. Now, each instruction must take as parameters: The address of the buffer you want to move from or move to. Usually, this is "@BUFFER", where BUFFER is the name of the buffer array. This must be a word address; this usually means that if you specify "@BUFFER", BUFFER must not be a byte array, but should rather be an integer, logical, or double array. The data segment number you want to move to or move from. The offset in the data segment at which you want to start the move. The length (in words) of the data to be moved. Note that if you specify an incorrect data segment number, invalid offset, invalid buffer address, or invalid length, HP will reward you with a system failure... The order of the parameters is NOT the same for both instructions. Rather, for each instruction you should put onto the stack the parameters that describe "where to move to", then the parameters that describe "where to move from", and then the length. In other words, the MFDS calling sequence would be TOS:=@BUFFER; << move to >> TOS:=DSEG'NUMBER; << move from >> TOS:=OFFSET'IN'DSEG; << move from >> TOS:=LENGTH; ASSEMBLE (MFDS 4); and the MTDS calling sequence would be TOS:=DSEG'NUMBER; << move to >> TOS:=OFFSET'IN'DSEG; << move to >> TOS:=@BUFFER; << move from >> TOS:=LENGTH; ASSEMBLE (MTDS 4); Also, don't forget that the instructions must be executed when in privileged mode. So, let's try a real-live application. Say that you want to write a program that prints out the session limit -- the maximum number of sessions that can be running at one time. Let's see how you could go about doing it. There's an old joke about a Chinese recipe for broiled rabbit -- the recipe starts with "first, catch the rabbit". We must first find out where the session limit is stored. Well, the session limit is logically job information ("job" in this context pertains to both batch jobs and online sessions), so it would most reasonably be in the "JOB INFORMATION" chapter of the System Tables manual -- Chapter 8. In fact, on page 8-??? is the layout of a table called the JMAT (Job MAster Table), and lo and behold, there in word 8 (MPE IV) or in word 10 (MPE V), is the session limit. So far, so good. And, what's more, on that very page (or barring that, in chapter 2) it says that the JMAT is data segment number (they probably say "DST Entry assignments") 25. So, the session limit is word 8 (MPE IV) or word 10 (MPE V) of data segment number 25. We know that it's one word long, so we can write the following program: $CONTROL NOSOURCE, USLINIT << :PREP me with ;CAP=PM >> BEGIN INTRINSIC GETPRIVMODE, GETUSERMODE, QUIT; INTEGER SESSION'LIMIT; GETPRIVMODE; TOS:=@SESSION'LIMIT; << Move to the SESSION'LIMIT variable >> TOS:=25; << Move from data segment 25 >> TOS:=8; << MPE IV; in MPE V, use 10 >> TOS:=1; << Move 1 word >> ASSEMBLE (MFDS 4); << Move From Data Segment >> GETUSERMODE; QUIT (SESSION'LIMIT); END. When you run this program, it'll get the session limit, and call QUIT with the specified parameter. I decided to call QUIT because QUIT will print the parameter and it's easier than calling PRINT and ASCII (I'm actually used to using my own SPL output package, which I described in the "Winning at MPE" column of the DEC 1983 through MAR 1984 Interact magazines; it makes numeric formatting much easier). Now, you know for sure whether or not this program works because you can do a :SHOWJOB and see the real session limit. However, if you tried to move something from a data segment and wasn't sure whether your program was working right or not, you could use privileged mode DEBUG to check it out. The command you'd use is "DDA" (Display DAta segment), and you'd enter "DDA dsegnumber+offset,length". Thus, to check out the above program, you can type: :DEBUG *DEBUG* PRIV.xx.xx ?DDA#25+#8 <<or #10 for MPE V>>,1,I DA31+10 +xxxxx ?E The "xxx"s stand for the values that depend on your system configuration -- the "+xxxxx" that comes out on the same line as "DA31+10" (or "DA31+12" in MPE V systems) is your session limit. Note that DEBUG echoes "DA31+10" -- 31 is the octal representation of decimal 25, and 10 is the octal for decimal 8. So that's really all there is to it -- figure out the data segment number to which to move from/to (which is either a constant given in the System Tables manual or a variable kept in some other system table) and the address within the data segment to move from/to, and then issue the appropriate MFDS or MTDS instruction. It's really rather simple, and it's a shame that HP doesn't explain it anywhere except a remote corner of the Machine Instruction Set Manual (and even there, not too well). It would have made sense for HP to describe this -- in fact, describe everything mentioned in this paper -- in the System Tables Manual or even a more widely distributed document. Oh well, I guess that's just not the "HP way". However, there is one gigantic problem with the above approach -- even if you're just doing MFDSs, passing an incorrect parameter will crash the system. Even if you write a procedure that does the "TOS:="s and the ASSEMBLEs, thus avoiding typographical errors, you're still going to have many bugs in your program, and having the system crash for each one of them is not a very appealing prospect (in fact, a very appalling prospect). This is especially a problem if you're not accessing permanent system tables (like the JMAT) but rather more ephemeral tables like other process's stacks. You could easily get the data segment number of another process's stack, and by the time you try to read it, the process (and the data segment) might be gone. The solution is to write procedures that get passed the MFDS/MTDS parameters, check them to ensure validity, and only do the MFDS or MTDS if the parameters are OK. That's what I'm going to talk about now, because writing code that uses MFDS and MTDS without having these safeguards (at least while debugging) is, in my opinion, grossly impractical. What kinds of errors can you have in your choice of parameters? Well one is that you're moving data from or into the wrong place. There's nothing any automatic checking can do about this -- that's a logical error that you'll have to find yourself. I only hope that these logical errors will happen to you mostly on MFDSs and not on MTDSs. There are several errors, though, that are automatically detectable: Negative length, data segment number, or offset. I'm not certain about this (and I'm not going to risk a system failure to try to find out), but maybe a negative length would mean "right-to-left" movement (like it does in the MOVE statement -- see the section on the MOVE statement in the SPL Reference Manual). However, since even if this were so, it wouldn't be a very useful option, a negative length, data segment number, or offset are pretty certainly errors. Note that negative buffer address is NOT an error -- remember, buffers in the DL-to-DB area will have negative addresses. Also, data segment number 0 is an error. Invalid data segment number. Not all data segment numbers are valid -- they can be up to 1023 in MPE IV and even more in MPE V, and there aren't always going to be that many real, allocated data segments. How could we check for this? Well, we'd have to look this up in another system table, the Data Segment Table (DST); it's described in System Tables Manual Chapter 2, and we'll talk more about it later. Buffer address out of bounds. This means that the starting address of the buffer is less than the DL register (i.e. the buffer starts in the PCBX or even lower, outside of your stack in somebody else's chunk of memory) or the ending address of the buffer (@BUFFER+LENGTH-1) is greater than the S register. Remember, this is privileged mode -- no bounds violation checking! Data segment offset out of bounds. This means that the offset is less than zero (already mentioned above) or offset+length-1 is greater than the current length of the data segment. The length of the data segment is also stored in the DST. So, our code would look something like: $CONTROL NOSOURCE, SEGMENT=DSEG'IO, SUBPROGRAM, USLINIT << Caller must be :PREPped with ;CAP=PM >> BEGIN DEFINE TURNOFFTRAPS = BEGIN PUSH (STATUS); TOS.(2:1):=0; SET (STATUS); END #; LOGICAL PROCEDURE DSEG'CHECKPARMS (DSEG'NUMBER, OFFSET, BUFFER, LENGTH); VALUE DSEG'NUMBER, OFFSET, LENGTH; INTEGER DSEG'NUMBER, OFFSET, LENGTH; ARRAY BUFFER; BEGIN << Returns TRUE if all parameters OK. >> INTRINSIC GETPRIVMODE; INTEGER POINTER DST = 2; << I'll talk more about this later >> INTEGER POINTER DL; << First, find out the address of DL -- the lowest valid stack address -- for use later on. >> PUSH (DL); << Push the value of the DL register onto the stack >> @DL:=TOS; << Pop it into our own pointer called "DL" >> << Now, get privileged mode for subsequent uses of "DST", the system table pointer we defined above (more about this later). Privileged mode should NOT be relinquished by this procedure (again, more about this later). >> TURNOFFTRAPS; GETPRIVMODE; << Now, check all the possible conditions >> IF << Is any of the parameters negative (or DST=0)? >> DSEG'NUMBER<=0 OR OFFSET<0 OR LENGTH<0 OR << Is DSEG'NUMBER greater than the maximum data segment #? >> DSEG'NUMBER>=DST(0) OR << Is DSEG'NUMBER invalid (indicated by having the length in its DST entry be 0)? >> 4*DST(4*DSEG'NUMBER).(3:13)=0 OR << Is the last data segment address to be moved (OFFSET+LENGTH-1) greater than the last valid data segment address (which is (data segment length - 1))? >> OFFSET+LENGTH-1>DST(4*DSEG'NUMBER).(3:13)*4-1 OR << Is the starting address of the buffer below DL? >> @BUFFER<@DL OR << Is the ending address of the buffer (@BUFFER(LENGTH-1)) above the greatest possible address, which is just one word below the first parameter to this procedure? >> @BUFFER(LENGTH-1)>@DSEG'NUMBER-1 THEN DSEG'CHECKPARMS:=FALSE ELSE DSEG'CHECKPARMS:=TRUE; END; << the procedure exit gets you back to user mode >> PROCEDURE DSEGREAD (DSEG'NUMBER, OFFSET, BUFFER, LENGTH); VALUE DSEG'NUMBER, OFFSET, LENGTH; INTEGER DSEG'NUMBER, OFFSET, LENGTH; ARRAY BUFFER; BEGIN << Reads LENGTH words from offset OFFSET of data segment # DSEG'NUMBER into the array BUFFER. Calls QUIT if any parameter is invalid. >> INTRINSIC GETPRIVMODE, QUIT; IF NOT DSEG'CHECKPARMS (DSEG'NUMBER, OFFSET, BUFFER, LENGTH) THEN QUIT (1717) ELSE BEGIN TOS:=@BUFFER; TOS:=DSEG'NUMBER; TOS:=OFFSET; TOS:=LENGTH; TURNOFFTRAPS; GETPRIVMODE; ASSEMBLE (MFDS 4); END; END; << the procedure exit gets you back to user mode >> PROCEDURE DSEGWRITE (DSEG'NUMBER, OFFSET, BUFFER, LENGTH); VALUE DSEG'NUMBER, OFFSET, LENGTH; INTEGER DSEG'NUMBER, OFFSET, LENGTH; ARRAY BUFFER; BEGIN << Writes LENGTH words from the array BUFFER to offset OFFSET of data segment # DSEG'NUMBER. Calls QUIT if any parameter is invalid. >> INTRINSIC GETPRIVMODE, QUIT; IF NOT DSEG'CHECKPARMS (DSEG'NUMBER, OFFSET, BUFFER, LENGTH) THEN QUIT (1718) ELSE BEGIN TOS:=DSEG'NUMBER; TOS:=OFFSET; TOS:=@BUFFER; TOS:=LENGTH; TURNOFFTRAPS; GETPRIVMODE; ASSEMBLE (MTDS 4); END; END; << the procedure exit gets you back to user mode >> END. I'll be the first to admit -- that's a lot of code. But with it, I am able to confidently test all my programs that read data segments (those that write other data segments can, of course, cause other nasty problems) on a production computer during working hours with no fear at all. In fact, I don't recall a single system failure in my past year of development, in which I've been writing and maintaining many programs like VESOFT's MPEX, LOGOFF, and others, all of which do heavy system tables work. When I did the conversion of these programs from MPE IV to MPE V, the first thing I did was to ensure that DSEGREAD works. Once that was done, despite the fact that there were bugs in the programs that it took several hours to iron out, there were no system failures. Now, a couple of comments about the above code are in order: * First, note that the procedures abort whenever an incorrect parameter is passed -- why? Well, an incorrect parameter is almost guaranteed to be caused by a program bug (by definition). What's the use of going on, doing other things, when you know that due to a problem bug, your program couldn't read or write data that it may be relying on? You could, of course, have the procedure return a logical flag, but then you'd have to check the flag every time you call it and probably wind up calling QUIT anyway. A good idea, however, is to somehow get the procedure to indicate the place they aborted and the cause of their abort -- negative parameter, invalid data segment number, bad offset, etc. * Second, note that all three procedures get privileged mode AND NEVER GET USER MODE! This is because whenever a procedure is exited, the mode is always reset to the mode of the caller; so, we don't need to get user mode explicitly. What's more, if we try to, it could actually hurt because it will cause an abort if the calling procedure itself is privileged. MPE does not permit code executing in user mode to exit to code executing in privileged mode. So, getting user mode explicitly is both unnecessary and undesirable. * Third, what's this "TURNOFFTRAPS" nonsense? Well, once upon a time (a long, long time ago, in a galaxy far, far away but really in Cupertino), I was having problems with my privileged code aborting. Well, a friendly HP person suggested that I turn off arithmetic traps (things like INTEGER OVERFLOW and others) before going into privileged mode, and lo and behold! all was well. Apparently, some privileged operations require arithmetic traps to be off (although why I'll never know), so to be on the safe side, I always turn them off before going into privileged mode. Like privileged mode, an exit from a procedure resets the setting of the arithmetic traps to what it was in the calling procedure. Note that some people (including Stan Sieler, an ex-HPer now at Allegro Consultants, who's likely to know about these things) believe that you only need to TURNOFFTRAPS before calling a system internal procedure (ATTACHIO, EXCHANGEDB, GETSIR, etc.) and not when you're just executing a privileged instruction (MFDS, LST, etc.). They may be right -- try it both ways and see. * Finally, let me just caution you that the above checks are NOT foolproof. Not only will they allow you to write incorrect data into data segments -- they'll make certain that the data segment number's correct but there's no way they could check the data -- but also, if you read or write a segment that is still around but is being deleted (or contracted), the procedure might check the segment while it's still around, see that all's well, and then do the MFDS/MTDS after the segment is gone. In this case, the system will indeed crash. I have NEVER gotten this to happen, but it's possible -- the only way to completely avoid it is to make sure that there are no bugs in your programs. Good luck... USING MFDS/MTDS TO ACCESS THE PCBX AREA When I was talking about accessing the DL-negative area, I mentioned that there was a way of doing it that I liked better than the one I presented. Well, here it is. Remember, your stack is a data segment like any other segment. You can access it using MFDS just as easily (or as difficultly) as, say, the PCB, or somebody else's stack. All you have to do is find its data segment number, which is not very hard, since it is in your process's PCB (that's Process Control Block -- a very useful system table) entry. Then, to read the PXGLOB, just do a DSEGREAD from your stack segment, location 0, length 8 (MPE IV) or 12 (MPE V). To get the PXFIXED, just do a DSEGREAD from your stack segment, with a starting location of 8 (MPE IV) or 12 (MPE V) -- just above the PXGLOB. To get the PXFILE, you have to get the PXGLOB, get the address of DL relative to the start of the data segment (that's PXGLOB cell 0), and then get the PXFILE address from DL-3. So, just write the following procedure: INTEGER PROCEDURE MYSTACK; BEGIN << Returns the data segment number of the process's stack >> INTRINSIC GETPRIVMODE; INTEGER POINTER << see below for more info on this construct >> PCB = 3; << SL procedure that returns the process's PIN >> INTEGER PROCEDURE MYPIN; OPTION EXTERNAL; TURNOFFTRAPS; GETPRIVMODE; MYSTACK:= << MPE IV: >> PCB (16*MYPIN+3).(1:10); << MPE V: >> PCB (21*MYPIN+3).(2:14); END; Note the use of the MYPIN procedure -- that's how you get your Process Identification Number (PIN), which is also the number of your PCB entry. Now, you can just call DSEGREAD, pass to it MYSTACK, and presto! you're reading your own stack. "Now, Eugene," you must be saying, "why would you want do a damn fool thing like that? There you have the DL negative area, easy as pie to access using pointers, and you insist on using an entirely different strategy. Why bother?" Well, the answer is really quite simple -- I'm lazy. I don't want to have to write and remember two sets of procedures -- DSEGREAD/DSEGWRITE and PCBXREAD/PCBXWRITE. I'd rather have one set that I would use to read either a data segment or the PCBX, and one way of looking at things -- everything's just another data segment, including the PCBX. System tables are complicated things, and any bit of conceptual simplification helps. AN EXAMPLE -- GETTING YOUR OWN SESSION NAME The perfect example of using MFDSs (or actually, DSEGREAD -- you should always call DSEGREAD, and never have an MFDS or MTDS anywhere else in your code) is getting your own job/session name. This is a useful piece of information that no HP-supplied intrinsic gives you -- you can use it to identify users (e.g. there's one user called CLERK.PAYROLL, with session names JOE, SUSAN, etc. -- a technique we support and encourage with our VESOFT SECURITY/3000 product), provide accounting information, and whatever else. First, we have to find out where it is. It comes under the broad category of "JOB INFORMATION", and if you look into the System Tables Manual, Chapter 8, you'll find it as a field of the Job Information Table (also known as the JIT). But, the JIT isn't like other tables, like the PCB, DST, or JMAT, which have constant data segment numbers -- each session has its own JIT. Well, it turns out (and this is NOT easy to find out) that the data segment number of the JIT is stored in the PXGLOB. So, what you'd do is: * Get the PXGLOB (using either of the methods shown above of accessing your PCBX). * With the JIT data segment number from the PXGLOB, get the JIT. * Extract the session name from the JIT. To see the actual program, look at program 2 in Appendix 1. ACCESSING MEMORY-RESIDENT SYSTEM TABLES As if EXCHANGEDB and MFDS/MTDS weren't confusing enough, there's yet another way of accessing some system tables. Certain system tables (e.g. the PCB, DST, CST [Code Segment Table], etc.) are memory-resident -- they are always in main memory and are never swapped out to disc. These tables can be accessed using two special machine instructions called LST (Load from System Table) and SST (Store into System Table). This would be a moot point if not for the fact that SPL has an feature that uses these instructions to allow you to easily access memory-resident system tables. (Note that this feature has only been documented in the FEB 84 version of the SPL reference manual, p. 2-13, 7-18, and 7-19; all prior versions of the reference manual do not mention it.) In SPL, if you say "INTEGER POINTER name = number;", all subsequent references to "name(index)" will access the index'th word of the memory-resident table indicated by "number". Thus, if you declare "INTEGER POINTER TAB'PCB = 3;", saying "I:=2+TAB'PCB(1);" will set I to 2 plus the value of word 1 of the PCB. Or, if you say "TAB'PCB(SIZE'PCB*PIN+7):=24;", it will move 24 to the SIZE'PCB*PIN+7th word of the PCB. This means that to access these tables, you need not do an MFDS or MTDS -- which are somewhat slower and also more cumbersome to call -- but rather just treat the table as if it were an array that you could index just like an ordinary array. Several things must be noted about these constructs: * MEMORY-RESIDENT TABLE IDENTIFIERS (the numbers that you put after the "=" in an INTEGER POINTER declaration) ARE NOT THE SAME AS DATA SEGMENT NUMBERS! In the case of the PCB, the data segment number and the memory-resident table identifier happen to be the same (3). For other tables, this may not be the case. The way to determine a table's memory-resident table identifier is to look into Chapter 1 of the System Tables Manual where the System Global (SYSGLOB) area is described. If the Nth word is indicated as containing the address of (or pointer to) a system table, then N is that table's identifying number. * Memory-resident tables can only be accessed in privileged mode. That means that whenever you want to use a memory table pointer (declared using the "INTEGER POINTER name = number" construct), you must be in privileged mode. * Memory-resident table pointers can only be used when indexed, and then only in expressions or in assignment statements. They can not be used in MOVE or SCAN statements, as by-reference parameters in procedure calls, or without an index. If you want to move several words from a system table, use either an MFDS or memory-resident table pointers and a FOR loop. * I haven't the foggiest notion of what would happen if you specify an invalid index when using a memory-resident table pointer (i.e. a negative index or one that is greater than the size of the table). I'd guess that if you pass an index greater than the table size, it would just get you data from whatever happens to be in memory at the system table base + the index, but if you specify a negative index or an index that would cause the effective address (the system table base + the index) to be outside of the memory bank in which the system table is located, be prepared for a system failure. * Remember that relatively few system tables are accessible in this way. Personally, I do not often use this method of system table access. For one, as I said before, I like to consistently use one approach, and not confuse myself with many different ones -- my favorite is MFDS/MTDS. Furthermore, since I have allocated a special array for each system table, and have DEFINEs that access that array, I usually end up having to copy an entire table entry anyway (not just a single word); however, if you use the alternative approach described above (i.e. having DEFINEs specify only the index into the entry without the array name), you wouldn't have to move the entire table entry. Sometimes, I do use memory-resident table pointers, primarily when speed is of the essence -- I've been told that LST and SST are faster than MFDS and MTDS. It's mostly a matter of personal preference -- use whichever approach you find most appropriate. An example of this approach could be the following procedure that determines whether it's running on MPE IV or MPE V: LOGICAL PROCEDURE MPE'V; BEGIN << result := TRUE if MPE V, FALSE if MPE IV >> INTRINSIC GETPRIVMODE; INTEGER POINTER TAB'PCB = 3; DEFINE PCB'ENTRY'LEN = TAB'PCB (1) #; EQUATE MPE'V'ENTRY'LEN = %25; TURNOFFTRAPS; GETPRIVMODE; IF PCB'ENTRY'LEN=MPE'V'ENTRY'LEN THEN MPE'V:=TRUE ELSE MPE'V:=FALSE; END; Note that the PCB table entry size changed from %20 in MPE IV to %25 in MPE V. Since the entry size is stored (in both MPE IV and MPE V) in word 1 of the PCB table, we can just look at that word and see if it's %25 or not. This procedure is exceptionally useful for writing programs that work on either MPE IV or MPE V, or at least abort if they're run on a version of MPE other than the one they were written for. If your program is written for MPE IV, it's much better to abort with a nice error message when run on MPE V rather than crashing the system. Note however that if you wanted to, you could perform the same task using MFDS. ABSOLUTE MEMORY LOCATIONS Some data is stored not in data segment-relative locations, but rather absolute memory addresses. For instance, there is an area in memory called "SYSGLOB" (which stands for SYStem GLOBal), which contains certain interesting pieces of information, like the current console device, the global allow mask, and lots of other goodies. It happens to be stored at a fixed address -- it goes from locations %1000 to %1377 in memory bank 0, and can be accessed using the "ABSOLUTE" construct of SPL. For instance to determine the system console logical device number (which is stored in location %74 of the SYSGLOB), you'd do something like: TURNOFFTRAPS; GETPRIVMODE; CONSOLE'LDEV:=ABSOLUTE (%1074); GETUSERMODE; TURNONTRAPS; Simple -- think of "ABSOLUTE" as an integer array whose 0th word is memory address 0 of bank 0. To store a value into this location, just say "ABSOLUTE (%1074) := NEW'CONSOLE'LDEV;". Note however that unlike a normal array, you can only read and write individual absolute locations; you can't do a MOVE or SCAN with an absolute address, nor can you pass it by reference to a procedure. In this respect, it's like the system table pointers discussed above. To confirm your results, or to access absolute locations without a program, you can use privileged mode DEBUG's "DA" (Display Absolute) command: :DEBUG *DEBUG* PRIV.xx.xx ?DA1074,I A1074 +00020 ?E Also note that DEBUG has a "DSY" (Display SYstem global) command, with "DSY n" being equivalent to "DA 1000+n" -- it just gets you the nth word of SYSGLOB. One word of caution: ABSOLUTE can only be used to access memory locations with addresses 0 to 65535 -- those memory locations in the so-called "memory bank 0". HP has recently been on a campaign to reduce the amount of data that must be stored in bank 0 -- tables like the PCB, CST, DST, and others that used to be always in bank 0 in MPE IV are no longer always in bank 0 in MPE V. The upshot of this is that since the only things that can be accessed using ABSOLUTE are the ones that are guaranteed to be stored in bank 0, you should avoid using ABSOLUTE whenever a non-bank 0 dependent access method (e.g. MFDS) is available. DISC RESIDENT SYSTEM TABLES, PART I -- FILE LABELS Unlike process information, data segment information, and job information, which are stored in memory, file information -- the size of a file, its file code, lockword, location on disc, etc. -- can not be stored in memory because it has to stay around through system failures. Most of the important file information, including all of the stuff that all modes of :LISTF show, is stored on disc in "file labels". A file's file label is pointed to by its directory entry and occupies one sector (128 words) on disc. Since it's stored in disc rather than in memory, accessing it is rather different from accessing data segments. In order to access -- read or write -- a file's file label you must first find out the logical device number (LDEV) of the disc on which the file label resides and the sector number of the file label on that disc. This is by no means a trivial task, and in many instances is a lot more complicated than actually accessing the file label once you have this. Logical device numbers and sector addresses are stored in various system tables, in a variety of formats (which we'll discuss later). However, for the duration of this discussion, we'll assume that you are trying to read or write the file label of a file that you have opened; that way, you can figure out the file label address by calling FGETINFO (or FFILEINFO mode 19). Now you have the file label address -- a doubleword, containing the LDEV in the 8 most significant bits and the sector number in the 24 least significant bits (a format we'll call type L). All you need to do is to perform the actual I/O. The fundamental file label I/O procedure is, not surprisingly, FLABIO: INTEGER PROCEDURE FLABIO (LDEV, ADDRESS, CMD, FLAB); VALUE LDEV, ADDRESS, CMD; INTEGER LDEV, CMD; DOUBLE ADDRESS; INTEGER ARRAY FLAB; OPTION EXTERNAL, UNCALLABLE; << must be called from PM >> Its operation is quite simple -- you pass to it the LDEV and sector address, CMD=0 to read or CMD=1 to write, and the 128-word array into which the file label is to be read or from which it is to be written. The result returned is 0 if all is OK, 1 if there was a "soft error" doing the I/O (I think that this means that the check sum in the file label being read is wrong), and 2 if there was a "hard error" doing the I/O (I think that this means a physical I/O error). Simple enough, once you have the file label's address. So, we can write the following procedure: << Read the file label of the file opened as FNUM into the array FLAB; return true if all OK, FALSE if failed. >> LOGICAL PROCEDURE FREADFILELABEL (FNUM, FLAB); VALUE FNUM; INTEGER FNUM; ARRAY FLAB; BEGIN INTRINSIC FGETINFO, GETPRIVMODE; INTEGER LDEV; DOUBLE ADDRESS; INTEGER ARRAY ADDRESS'I(*)=ADDRESS; INTEGER PROCEDURE FLABIO (LDEV, ADDRESS, CMD, FLAB); VALUE LDEV, ADDRESS, CMD; INTEGER LDEV, CMD; DOUBLE ADDRESS; INTEGER ARRAY FLAB; OPTION EXTERNAL, UNCALLABLE; << To be really safe, you should have code here that checks to make sure that reading 128 words into FLAB won't cause a bounds violation. For more info on this, see below in the discussion of disc address validity checking. >> FFILEINFO (FNUM, 19, ADDRESS); IF <> OR ADDRESS=0 THEN << either file not opened or not on disc (address=0) >> FREADFILELABEL:=FALSE ELSE BEGIN LDEV:=ADDRESS'I(0).(0:8); << high-order 8 bits >> ADDRESS'I(0).(0:8):=0; << ADDRESS now is sector address >> TURNOFFTRAPS; << See above (MFDS) to learn about this >> GETPRIVMODE; IF FLABIO (LDEV, ADDRESS, 0 << read >>, FLAB) <> 0 THEN FREADFILELABEL:=FALSE ELSE FREADFILELABEL:=TRUE; << no GETUSERMODE -- also see above to learn about this >> TURNONTRAPS; END; END; Using this procedure, you can now read the file label of any file you have opened, and find out a lot of information that FGETINFO and FFILEINFO won't give you -- things like the file creation date, last access date, last modification date, the file's extent map, lockword, and other useful things. A similar procedure can be written to write out an open file's file label -- just be sure that you don't use to change things that are also kept in memory-resident file tables that exist for open files (like the FCB, ACB, etc. -- see the System Tables Manual). Similarly, say that you want to read the file label of $OLDPASS *without* opening it. Why would you want to? Well, in the case of $OLDPASS you usually wouldn't (except if you want to save time) -- however, you might want to do this for other files that you can't open so easily, like spool files or other people's temporary files or $OLDPASSes. The first thing you'd have to do is to find out where the file label address for $OLDPASS is located. When you can recite this from memory, you are truly a superior system programmer -- mere mortals would have to scan through the System Tables Manual, make an educated guess or two, and find it in the Job Information Table (chapter 8), words 36 and 37. Looks simple enough -- there is your file label address; all you have to do is extract the LDEV from the 8 high-order bits, the sector address from the remainder, and call FLABIO. Right? Wrong. The fundamental means of doing the I/O are still the same -- you get the LDEV, get the sector address, and call FLABIO. However, what is stored in the high-order 8 bits of words 36 and 37 of the JIT is NOT (!!!) the logical device number of the disc. The $OLDPASS address in the JIT is what I call the "type V address" (as opposed to the "type L address" described earlier) -- its high-order 8 bits are the so-called "volume table index" of the disc on which the sector address is contained. So, you have a problem. You are trying to get the logical device number of the disc on which the file label is located and the file label's sector address. You have the file label's sector address, but instead of the LDEV, you have a volume table index (VTABX). What you need to do is to convert the VTABX into LDEV, and to do this you use a system internal procedure called LUN: INTEGER PROCEDURE LUN (VTABX, MVTABX); VALUE VTABX, MVTABX; INTEGER VTABX, MVTABX; OPTION EXTERNAL, UNCALLABLE; What LUN (which I suspect stands for "Logical Unit Number", another name for LDEV) does is take a VTABX and a MVTABX (Mounted Volume Table Index) and return the LDEV which they describe. Where do you get the MVTABX? Well, in this case, it is also (fortunately) stored in the JIT, in word 57. So, to read $OLDPASS's file label, you'd do the following: * First, read the JIT into an array (you can use the DSEGREAD procedure we described earlier to do this). * Then, take the $OLDPASS type V address (words 36 and 37), and separate out the VTABX (high-order 8 bits) and the sector number (low-order 24 bits). * If $OLDPASS's address is 0, GO NO FURTHER -- it means that no $OLDPASS file exists. * Call LUN, passing to it the VTABX and the MVTABX (from word 57 of the JIT). * Make a prayer that you did everything right ("Oh Lord, watch over this humble program and prevent it from crashing the system"). * Call FLABIO, passing to it the LDEV you got from LUN and the sector address you got from the low-order 24 bits of words 36 and 37 of the JIT. * I don't suppose I need to tell you to experiment with CMD=0 (read), not CMD=1 (write)... Voila! All you ever wanted to know about FLABIOing in ten pages or less... Incidentally, you're probably wondering what all this VTABX and MVTABX nonsense is about. Well, it was implemented to support Private Volumes, in which the logical device number of a disc is not known until the disc is actually mounted -- thus, the directory on the disc would have to contain not LDEVs, but VTABXs. If you expect your program to NEVER need to be run on Private Volumes, you can forget about MVTABXs, and just call LUN with an MVTABX of 0. However, you still have to call LUN, because even if your system never even smelled a private volume, the VTABXs need not correspond to the LDEVs. If you intend to use FLABIO often in your programs, I would suggest that you write the following procedures: * FLABREAD and FLABWRITE, which take a type L (LDEV and sector) address, and either read into or write from a user-specified array. For consistency's sake, they should return a file system error number (e.g. FLABIO error 1 would map into file system error 108 [INVALID FILE LABEL]; error 2 would map into file system error 47 [I/O ERROR ON FILE LABEL]). Also, since passing an out-of-range LDEV or sector to FLABIO will cause a system failure, this procedure can check the address using some tricks I'll talk about presently. * V'TO'L'ADDRESS, which takes a type V (VTABX and sector) address and a MVTABX, and returns the type L address. This would call LUN to convert the VTABX and MVTABX into the LDEV, and would allow you to easily read or write a file label given a type V address: FLABREAD (V'TO'L'ADDRESS (V'ADDRESS, MVTABX), FLAB); * FNUM'TO'L'ADDRESS, which takes the file number of an open file and returns the type L address. Simply calls FFILEINFO or FGETINFO. Again, makes reading/writing file labels given an FNUM easier: FLABREAD (FNUM'TO'L'ADDRESS (FNUM), FLAB); * Finally, FILE'TO'L'ADDRESS, which takes a filename and returns the type L address. The simplest way to do this is to FOPEN the file, call FNUM'TO'L'ADDRESS, and FCLOSE the file. A far more difficult approach, which however is faster and doesn't care about file security, lockwords, or whether the file is already opened, is to get the file label address directly from the directory. For this, you'd have to call the DIRECFIND procedure to get the sector address from the directory, call DIRECFIND again to get the directory entry for the group in which the file resides (to get the group's MVTABX), and then call V'TO'L'ADDRESS to convert the type V address that is stored in the directory to a type L address. For the beginning, stick to the simple approach -- the directory is probably worth a whole paper dedicated to it alone, and will not be elaborated further in this document. The algorithm of checking a type L address for validity is not trivial, but still well worth implementing (unless you want to get a reputation as "Mike 'System Failure' Johnson"): * Call the CHECKDISC procedure: PROCEDURE CHECKDISC (LDEV, STATUS); VALUE LDEV; INTEGER LDEV; LOGICAL STATUS; OPTION EXTERNAL, UNCALLABLE; Pass to it the LDEV, and make sure that the low-order 3 bits (bits .(13:3)) of STATUS are all 0 -- if bit 15 is set, this means the LDEV is out of range; if bit 14 is set, the LDEV is not configured; if bit 13 is set, the LDEV is not a disc. If none of these bits is set, you know you have a good LDEV. * Call the DISCSIZE procedure: DOUBLE PROCEDURE DISCSIZE (LDEV); VALUE LDEV; INTEGER LDEV; OPTION EXTERNAL, UNCALLABLE; Pass to it the LDEV, and make sure that the sector address is greater than or equal to 0 and less than the number DISCSIZE returned to you (which is the number of sectors on the device). * If you're a general-purpose procedure, check the buffer address passed to you to make sure that it's within bounds, i.e. that its start is at DL or above, and that its 127th word is at Q or below. Remember that bounds violations are not checked for in privileged mode, and trying to write to an out-of-bounds address would cause whatever is at that address (i.e. in someone else's space) to be over-written. * If either check failed, abort -- you've got a bad address, which if used in an FLABIO call would crash the system. If you want to check the results of your programs, or dabble with file labels without writing a program, there are several utilities you can use: * For some specific file label retrieval and modification tasks, VESOFT's own MPEX, which allows you to easily look at files' creator ids, access/modify/creation dates, etc., and modify many file attributes (like creator id). * HP's DISKED2, which allows you to read and write arbitrary disc locations. If you intend to modify file labels, make sure that you are either using the ">FILE" command of DISKED2 to get to the file label's sector (rather than just a >DISC and >MODIFY) or set the file label's checksum (word 34) to 0. If you use >DISC and >MODIFY but do not set the checksum to 0, the file system will not update the checksum and will think that the file label is corrupted. * Privileged mode DEBUG, which has a "DV" command (Display Virtual memory, a misnomer -- really displays an arbitrary disc sector). Note that the syntax of this command, especially when the sector address is more than 32768, is non-trivial; see the DEBUG Manual. * FLUTIL3, a contributed program which allows you to read and write the file label. It's easier to use than DISKED2, and is also safe for modifying file labels. Finally, a caveat to the user who knows about and uses ATTACHIO: resist the temptation to do file label I/Os using ATTACHIO instead of FLABIO. On reads, calling FLABIO is better because it checks the file label's check sum, letting you know if the file label appears corrupted. On writes, it is IMPERATIVE that you call FLABIO, because otherwise the check sum will not be updated, and next time the system wants to read the file label, it will think that it's corrupted. As a final present to all you file label hunters out there, the following table should tell you how to get file addresses: * Permanent files -- FOPEN the file and call FGETINFO/FFILEINFO (type L address); if you want to be fancy, call DIRECFIND to get the file's directory entry (type V address). * Your session's temporary files -- FOPEN/FGET(FILE)INFO (type L address), or get type V address from your JDT (described in System Tables Manual Chapter 8; pointed to by the PXGLOB, which is also described in Chapter 8). * Your session's $OLDPASS -- FOPEN/FGET(FILE)INFO (type L address), or get type V address from your JDT (described in System Tables Manual Chapter 8; pointed to by the PXGLOB, which is also described in Chapter 8). * Other sessions' temporary files and $OLDPASSes -- type V addresses stored in each session's JIT or JDT. * Spool files -- IDD or ODD system tables (System Tables Manual Chapter 14); type L. Good luck, and happy hunting. DISC RESIDENT SYSTEM TABLES, PART II -- USING ATTACHIO Other information besides file labels is also stored on discs -- things like disc labels, defective track tables, the contents of files themselves, and so on. To get at them, you have to use a privileged system internal procedure called ATTACHIO. ATTACHIO is *the* primitive I/O procedure. You can use it to do any arbitrary operation against any arbitrary device, from reading and writing discs to reading, writing, and doing various control functions to non-disc devices. All file system I/O functions eventually end up as ATTACHIO calls. The calling sequence for ATTACHIO is: DOUBLE PROCEDURE ATTACHIO (LDEV,QMISC,DSTX,ADDR,FUNC,CNT,P1,P2,FLAG); VALUE LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG; INTEGER LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG; OPTION EXTERNAL, UNCALLABLE; This is quite a mouthful (just what *is* a QMISC?). Fortunately, however, for discs these parameters are rather simple: * LDEV: This is self-explanatory. I'm sure I need not tell you that passing an incorrect LDEV will cause a system failure (#206, to be precise). * QMISC: 0. QMISC could have a lot of different values for non-disc devices, but for disc devices 0 will do quite well. * DSTX: Technically, the data segment number of the segment to/from which the read/write is to take place. 0 means your stack, and this is the value you'd use most often. * ADDR: The address (within the data segment indicated by DSTX) to/from which the read/write is to take place. When DSTX = 0 (i.e. your stack), this is the ordinary word address of the array which you're using for your I/O. Don't forget to have room for as much as you want to read/write in the array, since like all other privileged internal procedures, this one doesn't do ANY bounds checking. Also note that if you're reading into a stack array, you should be sure to pass the ADDRESS of the array (i.e. "@arrayname"). "ATTACHIO (1,0,0,@FOO,0,10,ADDR'HI,ADDR'LO,1)" will read into the array FOO; if you omit the "@" before the FOO, you will read into the stack address pointed to by the 0th word of FOO (which is probably not what you want). * FUNC: Curiously enough, the function. 0 means read, 1 means write. Try something else. I dare you. * CNT: If positive, this is the number of words to read or write; if negative, the absolute value of this is the number of bytes to read or write. Thus, 10 means 10 words; -20 means the same thing (20 bytes). * P1: The high order word of the sector number for the I/O. Don't forget that all reads and all writes must start at a sector boundary (although CNT need not be a multiple of the sector size). * P2: The low order word of the sector number. If you have a double word address D, you can say INTEGER D0=D, D1=D+1; D0 will now refer to the high-order word of D and D1 to the low-order word of D. Or you can calculate the high-order word of D on the fly by saying "INTEGER(D&DLSR(16))" and the low-order word by saying "INTEGER(D)". * FLAG: 1. This means "blocked, wait until request is complete" -- the I/O system's equivalent of ordinary file I/O (as opposed to no-wait I/O). * Function returns a double-word: Word 0 bits 8:5: qualifying status for the I/O; this further describes the result of the operation. The general status (see below) is usually all you need to find out what happened. Word 0 bits 13:3: general status of the I/O: 0: Pending. I don't think this should ever happen with flags = 1. 1: Normal. 2: End of file. I can't see how this could happen on disc (remember, ATTACHIO knows nothing about disc files). 3: Unusual condition. The qualifying status probably describes this better -- try figuring it out. 4: Irrecoverable error. Again, look at the qualifying status if you can figure it out. Word 1: number of words (positive) or bytes (negative) transferred. Incredibly, that's all there is to it. Once you've got an LDEV and a sector address, just plug them into the right parameters, and call away! Of course, remember to enter privileged mode and turn off traps (just as you would with FLABIO or an MFDS/MTDS). Also, remember that many disc addresses (the type V addresses I mentioned in the ACCESSING FILE LABELS CHAPTER) contain not an LDEV, but rather a VTABX which you'll have to convert into an LDEV. But, once you get all these preliminaries out of the way, calling ATTACHIO is just as easy as calling FLABIO. For instance, say that we want to determine the number of defective tracks on a given LDEV. First, as always, we have to find out where this number is stored. System tables Chapter 3 contains a lot of useful information on disc format, including the fact that sector 1 of every disc contains the disc's defective track table, and word 0 of the sector contains the actual number of defective tracks. Thus, our program would look something like: INTEGER PROCEDURE NUM'DEF'TRACKS (LDEV); VALUE LDEV; INTEGER LDEV; BEGIN INTRINSIC GETPRIVMODE; EQUATE SIZE'DEF'TRK'TAB = 128; ARRAY DEF'TRK'TAB (0:SIZE'DEF'TRK'TAB-1); DEFINE TAB'NUM'DEF'TRACKS = DEF'TRK'TAB (0) #; INTEGER DISC'STATUS; DEFINE LDEV'IS'DISC = (DISC'STATUS.(13:3) = 0) #; EQUATE FUNC'READ = 0; << ATTACHIO read function >> EQUATE DISC'IO'FLAGS = 1; << ATTACHIO flags for a disc I/O >> PROCEDURE CHECKDISC (LDEV, STATUS); VALUE LDEV; INTEGER LDEV; LOGICAL STATUS; OPTION EXTERNAL, UNCALLABLE; DOUBLE PROCEDURE ATTACHIO (LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG); VALUE LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG; INTEGER LDEV, QMISC, DSTX, ADDR, FUNC, CNT, P1, P2, FLAG; OPTION EXTERNAL, UNCALLABLE; TURNOFFTRAPS; << See MFDS chapter for a discussion of this >> GETPRIVMODE; CHECKDISC (LDEV, DISC'STATUS); IF NOT LDEV'IS'DISC THEN NUM'DEF'TRACKS:=-1 << error, not a valid sic >> ELSE BEGIN ATTACHIO (LDEV, <<qmisc>> 0, << Amazingly dreadful things will happen if we leave off the "@" in "@DEF'TRK'TAB >> <<dstx>> 0, <<address>> @DEF'TRK'TAB, FUNC'READ, <<cnt>> SIZE'DEF'TRK'TAB, <<sector address:>> 0, 1, DISC'IO'FLAGS); NUM'DEF'TRACKS:=TAB'NUM'DEF'TRACKS; END; END; That's all. Relatively simple, and because of the CHECKDISC call that ensures the LDEV is in range, configured, and a disc, no danger of system failure. Of course, this particular ATTACHIO calling sequence is only relevant to discs; terminals, printers, tape drives, and other devices have their own parameters for QMISC, FUNC, P1, P2, and FLAGS, which I will not explain here for lack of space and because they really are not relevant to system table access. SIRS, AND WHAT HAVE THEY DONE TO GET SUCH DEFERENCE? Whenever you are accessing (reading or writing) a system table that somebody else might be modifying, you have to beware of something being changed out from under you. For instance, say that you're reading the ODD -- the Output Device Directory, a rather odd name for the table that contains information on all the output spool files in the system. The ODD is structured as several linked lists of entries, one for each spooled device or device class. The way that you'd read it is that you'd read one entry, process it, get the address of the next entry from the current entry, get the next entry, process it, and so on. However, say that while you're processing this entry, the next entry in the list gets deleted. Then, when you try to get the next entry from the location that is pointed to by the current entry, you'll get garbage. Even worse, say that you read the entry, process it, make some changes to the copy of the entry you keep in your stack, and then try to write it back out. Then, if the entry is deleted between the time you read it and write it back out, you stand the risk of over-writing whatever new entry might have been put there, probably with disastrous consequences. This is by no means a new problem -- it arises in file and database systems all the time. The general solution to this is some kind of locking mechanism, and that is precisely what SIRs are for. SIR stands for System Internal Resource. You may lock it by calling the privileged system internal procedure GETSIR and release it by calling the privileged system internal procedure RELSIR. Since all (in theory) processes that modify a system table must lock its SIR before starting the modification, if you lock the SIR you are guaranteed that no other process will modify the table until you unlock it. Most system tables that are usually modifiable by more than one process at a time (including things like the PCB, JMAT, ODD, etc., but excluding JITs and JDTs, because these are session-local tables and are thus very rarely concurrently accessed) have a SIR. The SIR assignments are listed in Chapter 5 of the System Tables Manual. Before using GETSIR and RELSIR, you have to declare them in your program as follows: INTEGER PROCEDURE GETSIR (SIR'NUMBER); VALUE SIR'NUMBER; INTEGER SIR'NUMBER; OPTION EXTERNAL, UNCALLABLE; PROCEDURE RELSIR (SIR'NUMBER, GETSIR'RESULT); VALUE SIR'NUMBER, GETSIR'RESULT; INTEGER SIR'NUMBER, GETSIR'RESULT; OPTION EXTERNAL, UNCALLABLE; When you want to get a SIR, you figure out its SIR number and pass it to GETSIR, saving the result returned by GETSIR. Then, to release it you call RELSIR, passing to it the SIR number and the result returned by GETSIR (this is very important!). For instance, say you want to go through the ODD, which is a linked list of entries, without being afraid that somebody might be in the middle of adding or deleting an entry, causing you to encounter a bad link. What you'd do is: * Find out the ODD SIR number from the System Tables Manual Chapter 5 (it's 4). * Lock the ODD SIR by calling GETSIR, saving the result in some variable -- have a special variable, reserved for this purpose (say, ODD'GETSIR'RESULT), that you are certain will not be accidentally changed -- I and J are definitely out of the question. * Do whatever you want to do with the ODD, remembering that if you abort for any reason before you unlock the SIR, THE SYSTEM WILL CRASH. Also, be sure that neither you nor any procedure you might call while you have the SIR tries to get a SIR whose priority number (see the discussion below) is lower than the ODD's. * When you're done, call RELSIR, passing to it the ODD SIR number and the value that GETSIR returned. * Breathe a sigh of relief that you haven't crashed the system. Similar procedures are used for locking other SIRs -- however, note that if you have more than one SIR locked at the same time, you must unlock them in the REVERSE ORDER of locking. Also note that you won't get any problems if you try to lock a SIR twice -- say, you lock it, and another procedure you call locks it again; just remember to unlock it twice, too (in the case both you and another procedure locks it, all that is necessary is that both the other procedure and you unlock it, again in the reverse order that you two locked it in). Watch out, though -- when you're dabbling with SIRs, there are a lot of possible pitfalls involved: * Locking a SIR is not just a nice thing to do that'll avoid problems for you. If you modify a table without first locking its SIR, you get everybody else in deep trouble, much like if you were modifying a shared file without first FLOCKing it. Unfortunately, unlike KSAM files and IMAGE databases (but like MPE flat files), no checking is done to ensure that you've gotten the right SIR -- it's your responsibility, and if you don't live up to it, may God have mercy on your system. * Like other resources, it's very easy to get into deadlock trouble when you're getting more than one SIR. Say that you get SIR A and then try to get SIR B. SIR B is locked by another process, so you're suspended until the other process unlocks the SIR. Meanwhile, the other process tries to get SIR A, and suspends waiting for you to unlock it! Both of you are waiting for each other to release a SIR, and you'll stay that way until the system is rebooted. What's worse, anyone else who tries to lock either SIR will be suspended, too, causing an ever-growing number of suspended problems, and making the system come to a grinding halt. To avoid this, there is a certain very fixed and unalterable SIR locking sequence that you MUST obey. It is described in Chapter 5 of the MPE V System Tables Manual, where each SIR is given a priority number, and you must never lock a SIR whose priority number is lower than that of one you already have locked. Note that a similar table in Chapter 5 of the MPE IV System Tables Manual is quite incomplete -- try to get the MPE V locking sequence (which, to the best of my knowledge, is the same as that for MPE IV, although it is described much better). When you're debugging SIR code that you're afraid might deadlock, it's a good idea to keep an OPT or a SOO process running on some other terminal. OPT and some versions of SOO have commands that allow you to see who is holding what SIRs -- this can help you find the cause of the deadlock. * Not only must you watch out to make sure that you lock all SIRs in the right order, you must also be certain that all your callers and called procedures do too. Thus, if you want to lock the FILE SIR (which must never be locked before the FMAVT [File Multi-Access Vector Table] SIR), you must be sure that you do not call any procedure that tries to lock the FMAVT SIR, since that would be a SIR locking sequence violation. Since virtually all file system intrinsics can under some conditions try to lock the FMAVT SIR, you must either lock neither the FILE SIR nor the FMAVT SIR or both the FILE SIR and the FMAVT SIR before calling a file system intrinsic. That goes for all other procedures you call -- you better know what SIRs they try to lock, and you better make sure that your SIR locking sequences are appropriate. Similarly, if a procedure that you write uses SIRs, you have to be sure that all your callers realize this and make sure that they either don't have any SIRs locked when they call you, or they are certain that your and their locking sequences mesh well. * If your process terminates itself for any reason (TERMINATE, QUIT, STACK OVERFLOW, whatever) while you have a SIR locked, you get a System Failure 314. You best be VERY careful. Note that you need only be afraid of your process terminating itself -- while you hold a SIR, you can't be killed from outside (say, by an :ABORTJOB); if someone tries, you'll keep on processing until you release your last SIR, and then you'll die. * Always disable break before locking a SIR. This is not because you can be :ABORTed from break -- if someone tries, you'll keep on going until you release your SIRs, and only then will the abort take hold. Rather, if you let a user hit break while you've got a SIR, the user might try to execute an MPE command that tries to get the same SIR, and you'll get a deadlock -- the command waiting for the SIR (which is held by your program) and your program waiting for MPE to :RESUME it (which it can't because it's waiting for a SIR). Deadlocks work in mysterious ways. * Never do any terminal I/O while you have a SIR locked. In fact, do not perform any task that you suspect might take a long time to finish -- like issuing a request for a tape, reading a possibly empty message file, etc. The reason for this is simple -- if a process that has a SIR suspends, everybody else who wants the SIR will suspend, too, thus effectively stopping the system until the SIR holder is re-activated. The last thing you want is for the system to come to a grinding halt because someone went on a coffee break while a SIR-holding program was expecting input from him. As I said, this is also relevant for terminal I/O because a control-S on the terminal will suspend the writing program until a control-Q is hit. * Always be sure that the GETSIR result you pass to a RELSIR is the result returned by the GETSIR that got the particular SIR involved. If not, three guesses as to what happens... * Always be sure that you unlock SIRs in the inverse order of locking them, under penalty of you-know-what. This is an array of warnings that should make the bravest programmer cringe. Unfortunately, if you want to modify a system table or even read system tables that you fear might change out from under you, you have to lock the SIR. So, how can you do this without fearing a system failure? * Try to avoid locking SIRs whenever possible. In MPEX, I used to lock the FMAVT SIR and the FILE SIR (the two SIRs you should lock when doing certain operations on files -- always lock them in that order) when I modify a file's file label. Recently, I've changed it so that I wouldn't modify the file label without first FOPENing the file exclusively -- that way, I'm guaranteed that no-one else (except possibly :STORE/:RESTORE) is dabbling with the file, and I no longer need to lock the SIR. This avoids risk of deadlocks and system failures. * When you need to get a SIR, release it as soon as possible. Every statement between a GETSIR and a RELSIR is just one more opportunity for a disastrous (system-crashing) abort to occur. * Write three procedures, SIR'GET, SIR'RELEASE, and SIR'RELEASE'ALL. - SIR'GET should take a SIR number, turn off break (see above), lock the SIR, and save the SIR number and the GETSIR result in a global array. This global array should be shared between these three procedures, and should actually be treated as a stack, with new entries being added to the end, and old entries being deleted (by SIR'RELEASE) in a last-in, first-out fashion. - SIR'RELEASE should take a SIR number, make sure that it's the most recently gotten SIR, get the GETSIR result from the last-added entry in the global array, and call RELSIR. - SIR'RELEASE'ALL shouldn't take any parameters, but should only release all the SIRs mentioned in the global array maintained by SIR'GET and SIR'RELEASE. It should be called by any procedure that wants to terminate or abort in any fashion. For instance, the DSEGREAD procedure that I talked about above does a MFDS, checking its parameters for validity first, and aborting if the parameters are invalid (to avoid a system failure). If you ever call it while you have a SIR locked, put a call to SIR'RELEASE'ALL into it right before it aborts. That way, if you have a bug and DSEGREAD aborts, it'll release all the SIRs you have locked before aborting, thus preventing a system failure. I use these procedures exclusively, almost never calling GETSIR and RELSIR outside them, and my MPEX (or any of VESOFT's other privileged mode-using programs) has never crashed the system in production by aborting while holding a SIR (in fact, they've never crashed a production system, period). Sometimes, we get a letter containing a PSCREEN of an abort caused by an MPEX bug (yes, even MPEX has bugs) with the message "DSEGREAD passed bad parameter, SIRs released", and I feel kind of proud -- sure, MPEX may have a bug, but even though it was trying to read a non-existent segment while it had a SIR locked, it just aborted with a nice message instead of crashing the system. The number one cause of user program-caused system failures is not carelessness, but laziness -- the user didn't spend the time to develop some simple utility procedures that would prevent minor bugs or typos from causing system failures. A final aside on the topic of SIRs -- sometimes (fortunately, very rarely) you may want to ensure that NOBODY else on the system is doing anything. Essentially, you want to turn off interrupts to make sure that until you turn them back on, you and you alone will use the CPU. This is kind of a "super-SIR" -- you don't just lock some table, you lock the entire system. I don't like this, and I've never felt the need to do this. For one, you can crash the system this way easy as pie -- from what I understand, any interrupt, including one caused by a request on your part to access a data segment that isn't currently in memory would cause a system failure. Furthermore, this isn't as good as a SIR in at least one way -- when you lock, say, the JMAT SIR, you can be sure that you will wait until anybody who locked the SIR to modify the JMAT releases it; that way, when you get it, you know that nobody else is dabbling with the table. This is not so for turning off interrupts -- the process which is modifying the table you want to modify may have just been swapped out, and the table might be in a temporarily incosistent state. Finally, when you turn off interrupts, you also turn off clock interrupts, and the system clock is essentially not running until you turn interrupts back on. As I said, I personally have never needed to do this, and don't foresee myself doing it in the future. However, if you feel up to it, there are two machine instructions that do this -- PSDB (PSeudo-DisaBle) and PSEB (PSeudo-EnaBle). PSDB disables process dispatching, thus assuring that you'll be the only one to run until you do a PSEB. Unfortunately, this may (or may not -- I'm not sure) mean that the system will crash if you try to access a data segment that is not in memory, or do any other kind of disc I/O. This means that about the only thing you can safely do between a PSDB and PSEB is to access system tables that you know are always memory-resident (e.g. CST, DST, PCB, etc.). If you have any luck doing this, let me know -- I'd like to see how this should really be done. SYSTEM TABLES AND SPL PROGRAMMING STYLE Everybody and his brother have their own opinions about programming. The last thing that I want to do is to enter this controversial and generally thankless field. However, in my experience with using system tables in SPL, I discovered certain useful programming guidelines that I want to mention in passing. The major problem in working effectively with system tables is that virtually nobody can remember just what the 5th bit of the 53rd word of an entry in data segment number 22 contains. Furthermore, it is rather cumbersome to always flip through the System Tables manual whenever you're writing or reading programs. Comments help some for program readability, but you'd still have to look at the manual while writing, and you also stand the risk of incorrect comments. What one really wants is record structures (like in PASCAL). You declare a data type PCB'ENTRY'TYPE, declare a data structure of this type called PCB'ENTRY, read the entry into PCB'ENTRY, and then just refer to the PCB entry fields as PCB'ENTRY.FATHER'PIN, PCB'ENTRY.PRIORITY, PCB'ENTRY.CS'QUEUE, etc. If you have another PCB entry you want to simultaneously work on, just put it into another data structure -- ANOTHER'PCB'ENTRY -- and use ANOTHER'PCB'ENTRY.FATHER'PIN, etc. Unfortunately, SPL does not have these kinds of record structures. However, they can be cleverly emulated. What I do is that I declare an array called PCB, and I set up a number of DEFINEs -- PCB'FATHER'PIN, PCB'PRIORITY, PCB'CS'QUEUE, etc. -- that refer to the appropriate fields of the array. Then, when the array contains a PCB entry, these DEFINEs can refer (on either the left or right side of an assignment statement) to the appropriate fields of the entry stored in the array. Note that these DEFINEs can refer to individual bits of the array as well as to entire words. For instance, one such definition of some of the PCB entries might look like: << Definitions of some fields of MPE IV PCB entries. >> EQUATE SIZE'PCB = 16; INTEGER ARRAY PCB(0:SIZE'PCB-1); DEFINE PCB'FATHER = PCB( 5).( 0: 8) #, PCB'PRIORITY = PCB(13).( 8: 8) #, PCB'CS'QUEUE = PCB(13).( 2: 1) #; Note that I also have an equate for the PCB size; I'd also have an equate for the PCB data segment number, the PCB System Internal Resource (SIR) number (if it had one), equates or defines for all interesting PCB constants (e.g. the possible values of the "process type" field), etc. What's more, I'd put this all into one $INCLUDE file so that it will be kept in one place and will thus be easily modifiable. Other tables are a bit more difficult. The JIT (Job Information Table), for instance, contains not just word (integer) and bit field values, but also character arrays and double integers. For that, I equivalence (using the "BYTE/DOUBLE ARRAY xxx(*)=yyy") a byte array and a double array to JIT, the main array (which is an integer array). Thus, the file looks something like: EQUATE SIZE'JIT=61; INTEGER ARRAY JIT(0:SIZE'JIT-1); BYTE ARRAY JIT'B(*)=JIT; DOUBLE ARRAY JIT'D(*)=JIT; DEFINE JIT'JOB'ID = JIT ( 9) #, JIT'MAIN'PIN = JIT (10). ( 8: 8) #, JIT'GROUP'SEC = JIT'D( 7) #, << Word 14 >> JIT'ACCT'NAME = JIT'B(32) #; << Word 16 >> A very few tables contain double integers that are not aligned on an even word boundary (e.g. word 14), but are rather on an odd word boundary (e.g. word 21), and thus can't easily be accessed as elements of a double array equivalenced to the main integer array. For them, you have to equivalence another double array to the 1st (as opposed to 0th, the default) element of the main array, and then reference them as elements of this array. For example, EQUATE SIZE'DIR'GROUP=41; INTEGER ARRAY DIR'GROUP(0:SIZE'DIR'GROUP-1); BYTE ARRAY DIR'GROUP'B(*)=DIR'GROUP; DOUBLE ARRAY DIR'GROUP'D(*)=DIR'GROUP; DOUBLE ARRAY DIR'GROUP'D'1(*)=DIR'GROUP(1); DEFINE DG'CPU'COUNT = DIR'GROUP'D'1(6) #; << Word 13 >> Another advantage of this scheme is that it makes it much easier to change your programs to work on a new release of MPE in which the format of a table has been changed -- sometimes you only have to change the $INCLUDE files and recompile. Also, it makes it surprisingly easy to have the same source code work with two different MPE versions (e.g. MPE IV and MPE V) -- just declare a compile-time switch, say, X5 that is ON when you're compiling the program to run on MPE V and OFF when you're compiling to run on MPE IV. Then, in all your $INCLUDE files that describe tables that are different in MPE IV and MPE V, just check this switch and depending on its value, declare either the MPE IV or MPE V layouts. For more information on compile-time switches, see the SPL Reference Manual. An example is the PCB: $IF X5=OFF << MPE IV >> EQUATE SIZE'PCB=16; INTEGER ARRAY PCB(0:SIZE'PCB-1); ... DEFINE PCB'STACK'DST = PCB( 3).( 1:10) #; ... $IF X5=ON << MPE V >> EQUATE SIZE'PCB=21; INTEGER ARRAY PCB(0:SIZE'PCB-1); ... DEFINE PCB'STACK'DST = PCB( 3).( 2:14) #; ... $IF The major problem of this method is that it requires the entry to be stored in a certain fixed place (say, the array PCB). If you have two entries that you want to manipulate at the same time, you're out of luck; if you have an entire array of entries that you want to look through, you have to step through the array, moving each entry to the array PCB before processing the entry. Also, if you want to access system tables using SPL memory-resident system table support constructs (see below), you have to move each word of the table into the array. One solution to this problem is to have the DEFINEs contain only the word number of the element in the table. For instance, for the JIT we might have: EQUATE SIZE'JIT=61; DEFINE JIT'JOB'ID = 9 #, JIT'ACCT'SEC = 12 #, JIT'ACCT'NAME = 32 #, << Byte field! >> JIT'ALLOW'1 = 40 #; Then, to get the job id, we'd read the JIT entry into anywhere we want to (say, the array MY'JIT'ENTRY), and access it as MY'JIT'ENTRY(JIT'JOB'ID). This will work not only for fields that are whole words, but also for bit fields: DEFINE JIT'MAX'PRI = 10).(0:8 #, JIT'MAIN'PIN = 10).(8:8 #; This way, MY'JIT'ENTRY(JIT'MAIN'PIN) would map to MY'JIT'ENTRY( 10).(0:8 ) -- the right value. Unfortunately, one of the drawbacks of this method is that if you want to retrieve a byte field or a double field, you can't just specify it by name -- you have to explicitly retrieve it as MY'JIT'ENTRY'B(JIT'ACCT'NAME). If you forget and enter MY'JIT'ENTRY(JIT'ACCT'NAME), you'll get a very wrong result. Also, this method requires more typing -- you have to enter both the field designator (e.g. JIT'ACCT'NAME) and the array neme. Both methods are acceptable approaches, each with its own advantages and disadvantages. Use whichever you prefer, or even your own, but keep in mind the following: * Use DEFINEs and EQUATEs (for field names, entry size, data segment numbers, SIR numbers, possible field values, etc.). If you use "magic numbers" (e.g. bits 12:3 of word 35) in your code, it'll only make it harder to write, read, and maintain. * Put DEFINEs and EQUATEs into an $INCLUDE file. That way, if you need to change something (because you made an error in putting it in in the first place or because it has changed in the new MPE release), you'll only have to change it in one place. AND NOW, FOR SOMETHING COMPLETELY DIFFERENT... Until now, I have been concentrating mostly on describing how system tables can be accessed. To do this, I've introduced some important tables like the PCB and JMAT, but have skimped on description of what other system tables may exist and what they may contain. Unfortunately, the System Tables Manual does not really explain anything except table formats. It'll tell you what the ODD looks like, but won't tell you thing one about how it fits into the overall system structure and what you need to know to access it. Once I've whetted your appetite by telling you how to access system tables, I feel obligated to present my concept of how all these tables fit together and where you should go to get a particular piece of data. As I said before, system tables are just the places where the operating system "keeps its stuff" -- where it stores information on all the objects it must manage. A convenient way to look at system tables is in terms of what objects they describe. Don't feel nervous if you don't understand the purpose or even the contents of some of the tables I mention below. Some of them are really arcane, and may not be worth much to you anyway. If you're just starting to learn the innards of the system, you might want to skip reading about everything except the simpler job-, process-, and file-oriented system tables. JOB-ORIENTED SYSTEM TABLES Almost all the work done on an HP3000 is done on behalf of a job submitted by the user. Note that whenever I say "job", I mean either a batch job or on-line session, since they are rather similar to the system. From the system's point of view, a job is a collection of processes. The Command Interpreter (CI) of a job is one such process. Any processes you create, either using the :RUN command or using the CREATE or CREATEPROCESS intrinsics, also belong to that job. Every job has an entry for it in the Job MAster Table (JMAT). This is a very simple table; it has a header entry, which contains system global information (like the number of jobs/sessions, the job/session limit, etc.), and one entry for each job on the system. Each of these entries contains some (but not all) information about the job -- its job name, user, account, group, terminal number (for sessions), the Process Identification Number of its main process, and so on. A :SHOWJOB takes virtually all of the information it prints from the JMAT -- just think of the JMAT as a :SHOWJOB stored in a machine-readable form. Since there's only one JMAT, it occupies a data segment with a fixed data segment number. If you want to know the data segment number or the page of the System Tables Manual on which the JMAT is described, just look at Appendix 2 of this paper. A job's JMAT entry, however, does not contain all the information about the job. Much of the remaining information -- CPU time, the capabilities of the user running the job, :ALLOW mask, etc. -- is kept in a table called the Job Information Table (JIT). There's one JIT for each session, and it's pointed to by the PXGLOB area (which we mentioned earlier) of the stack of each process that belongs to the job. To get to your JIT, you'd first find its data segment number from your PXGLOB, and then use an MFDS to read it. Just as the :SHOWJOB command reads the JMAT, the WHO intrinsic looks at your JIT; all the information it gives you -- capabilities, user, account, group, home group, local attributes, etc. -- all come from the JIT. The JIT is even easier to work with than the JMAT, being just a big record structure with a lot of fields (each of which is at least briefly mentioned in the System Tables Manual). Note that there's some duplication of data (user name, account name, group name, etc.) between the JMAT and the JIT. HP does some damnedest things. If you've been paying particularly close attention, you'll find that I lied. A job is more than just a collection of processes -- there are also some job-local entities, such as :FILE equations and temporary files, that have to be maintained for each job. These are kept in a Job Directory Table (JDT), of which there is one per session (also pointed to by the PXGLOBs of the processes belonging to that session). This table actually contains a header and five sub-tables: one for the job's temporary files, one for the :FILE equations, one for the JCWs, one for the job-local data segments, and one for the :CLINE (a datacomm command) equations. The header contains the pointer to each of the sub-tables, and each sub-table is in turn is a collection of variable-length entries. Thus, to find a particular :FILE equation, you'd get the JDT's data segment number from your PXGLOB, get the JDT-relative index of the :FILE equation table from the JDT header, and then go through the :FILE equation table entries until you find one with the name you're looking for. Not the most entertaining job in the world, but not too difficult, either. Finally, for completeness' sake I must mention two other, relatively unimportant, tables -- the Job CUtoff Table (JCUT) and the Job Process CouNt Table (JPCNT). The JCUT contains information used for aborting all jobs that have a CPU time limit. I haven't the foggiest notion of what the JPCNT is actually useful for, except for a little-used (if at all) device called Job SIRs. This is one table you can afford to ignore. Thus, to summarize, the structure of those system tables that pertain to jobs is roughly like the following: JMAT (a single data segment), which contains for each job The job number, job name, user, account, group, terminal number, etc. The Process Identification Number (PIN) of the job's main (CI) process One PXGLOB per process belonging to the job, which points to One JIT per job, which contains The job number, job name, user, account, group, home group, CPU time, :ALLOW mask, etc. One JDT per job, which contains information on the job's Temporary files (JTFD) :FILE equations (JFEQ) Data segments (JDSD) JCWs (JJCW) :CLINE equations (JLEQ) The JMAT entry referring to the job JCUT (stand-alone table), which contains Information on all jobs that have a CPU time limit JPCNT (stand-alone table), which contains A bunch of bits the reason for which I could never fathom PROCESS-ORIENTED SYSTEM TABLES Rather more important than the concept of a job is the concept of a process. A process is a single instance of a program running. It has its own data space and code (the code may be shared among several processes that are running the same program). All work done on the system is done on behalf of a process, some of which -- the user processes -- belong to jobs, and others -- the system processes -- do not. The master table that describes all processes is the Process Control Block table (PCB). It, like the JMAT, contains a header entry and one entry for each process. The process entry contains useful information like a process' priority, the data segment number of its stack, its family relationships (father, son, brother), wait flags (is it waiting for a SIR, a RIN, etc.), and more. A process' PIN (Process Identification Number) is nothing more than the number of the process' entry in the table. If you have a PIN, you can just multiply it by the PCB entry size, and do an MFDS of the entry from that location in the PCB (which has a fixed data segment number). Also like the JMAT, the PCB does not contain all the information worth knowing about a particular process. Much very useful information -- what files are open by the process, what its register values are, what traps it has enabled, where its JIT and JDT are, and so on -- is kept in the so-called PCB eXtension (PCBX), which is stored in the DL-negative area of the process' stack. The PCBX consists of * The PXGLOB, which is mostly the same for all processes in a job and contains global information like the job's JIT and JDT data segment numbers, the device on which the job is running, the index of the process' JMAT entry, etc. * The PXFIXED, which contains a variety of useful data, like the values of the process' DB and S registers (they have to be kept somewhere when the process gets swapped out), the addresses of whatever control-Y and other trap routines are set, the job number of the job the process is running, and so on. Like the PXGLOB, this is just one big record structure. * The PXFILE, which describes the files the process has open. Its structure is so complicated that it deserves a separate paper (which may or may not be forthcoming). Fortunately, the System Tables Manual is somewhat more lucid than usual with regard to the structure of the PXFILE (which isn't saying much), and you might be able to figure things out by reading it. But don't bet on it. This is about all there is to the structure of the process tables. Note the recurring concept of a master table that contains some information on all processes or jobs, and a table for each process or job that contains more detailed information about the particular process or job. You'll see this again when we get to logical devices and the I/O tables. Finally, two other, less useful, tables. The Process-Job cross REFerence table (PJXREF), whose format IS NOT GIVEN IN THE SYSTEM TABLES MANUAL, contains, for every process, its job number. The job number of process PIN is in the PINth word of this table. The Process-Process COMmunication table (PPCOM) contains information on any mail the process might have in its mailbox. Thus, to summarize, the process table structure looks like: PCB (a single data segment), which contains for each process The process' priority, wait flags, etc. The PINs of the process' father, son, and brother The data segment number of the process' stack PCBX (stored in the process' stack's DL-negative area), containing PXGLOB, which contains The data segment of the process' JIT and JDT The index of the process' JMAT entry Some other information (like the process' terminal) PXFIXED, which contains The job number, saved registers, trap addresses, etc. PXFILE, which contains Information on all the files used by the process PPCOM (a single data segment), which contains for each process Information on any mail message it might have in its mailbox PJXREF (a single data segment), which contains for each process Its job number; the job # of process PIN is stored in the PINth word of the table DEVICE-ORIENTED SYSTEM TABLES Many system tables describe devices -- discs, terminals, etc. Of all the parts of MPE, the I/O system is probably the most intricate and most confusing portion. Devices are referred to by their Logical DEVice numbers (LDEVs). Although this means there's such a thing as a physical device, us software people don't care about it. In fact, I'll use the terms device, logical device, and LDEV interchangeably (since everybody else does, anyway). Every device has an entry in the Logical Device Table (LDT) and the Logical-Physical Device Table (LPDT), both of which are single data segments with fixed data segment numbers. The LDT contains the device type, record with, main pin of owning process (for terminals and tapes), etc. Note that the second half of the LDT's data segment contains the so-called LDT eXtension (LDTX), which contains some more useful information. Like the LDT, the LDTX has one entry per LDEV. The start address of the LDTX can be deduced from finding out the number of configuring LDT entries from the LDT header entry and then calculating the index of the first word that isn't used by the LDT. The LPDT contains some other stuff, like the device subtype, state, some more flags, and the address (SYSGLOB-relative!) of the device's Device Information Table (DIT). The DIT contains various information about the device that is highly specific to the particular device type. The System Tables Manual has 50-odd pages describing the various DIT formats. It is not very entertaining reading, and will never compete with a well-written, say, telephone book. All device classes in the system are listed, along with the LDEVs they represent, in the Device Class Table (DCT), which contains one variable-length entry per device class. To find out, say, what devices the class TAPE contains, just traverse the table until you find an entry with a name of TAPE. Like the LDT, the DCT shares its data segment with another (less useful) table called the Terminal Descriptor Table (TDT). All the I/Os that are currently pending in the system have entries in the I/O Queue (IOQ, for non-disc devices) and Disc Request Queue (DRQ, for disc devices) tables. The last thing I want to do is expend more energy describing these truly complicated tables that most people will never use anyway. Finally, some more useless tables: the Interrupt Linkage Table (ILT), Driver Linkage Table (DLT), System BUFfers (SBUF), Terminal BUFfers (TBUF), and the Interrupt Control Stack (ICS). Definitely not your bread-and-butter programming stuff. If you don't know about them, you're none the worse for it. Finally, a recap of the I/O System tables: LDT (a single data segment), which contains for each device Device type, record size, main owner pin (for non-discs), etc. LPDT (a single data segment), which contains for each device Device sub-type, device status, various other flags, etc. The address of the device's DIT DCT (a single data segment), which contains for each device class The LDEVs that are in the class The type of class (discs, serial I/O devices, etc.) DIT (a chunk of memory for each LDEV, pointed to by a SYSGLOB-relative address), which contains a lot of device-dependent data IOQ, DRQ, SBUF, TBUF, DLT, ILT, ICS, LDTX, and TDT, which you'll have to look up in the System Tables Manual if you really want to be a big hit at cocktail parties FILE-ORIENTED SYSTEM TABLES The entity that everybody works most with is the file. Every file in the system has a File LABel (FLAB) kept in a sector on disc. Although this is not a data segment, it can properly be called a system table, since it contains system-maintained data. The file label is a veritable treasure-trove of information, containing everything you'd ever want to know about a closed file. All modes of :LISTF get their information straight from the file label. Getting to a file's file label, however, isn't very easy. The disc address of a file label may be stored: * for permanent files, in the system directory * for job temporary files, in the job's JDT * for spool files, in the Output Device Directory (ODD) or Input Device Directory (IDD) * for files that were FOPENed as new files and not yet FCLOSEd, in the File Control Block (FCB) belonging to that file For open files, the matter is somewhat more complicated. In addition to the permanent, relatively unchanging information that is kept in the FLAB for a closed file, the system must also keep track of an open file's current record pointer, current block number, buffers, etc. As I said before, the topic of open file tables is such a complex one that it deserves a paper of its own. All I'll do here is give a brief structural description: FLAB (stored on disc, one per file) contains The file name, code, size, extent map, etc. -- everything a :LISTF could give you The system directory contains entries that point to FLABs of permanent files Each job's JDT contains entries that point to FLABs of job temporary files The spool file directories IDD and ODD contain entries that point to FLABs of spool files Each process' Active File Table (AFT), stored in the process' PXFILE area, briefly describes all the files opened by the process (a file number is an index into this table) and points to each file's LACB and PACB For each open of a file, a Physical Access Control Block (PACB) describes the current state of the file -- the current record number, the buffers used for buffered file access, the number of physical and logical I/Os that have occurred, and so on. For files opened MULTI, or GMULTI, there's only one such PACB for all openers; for all other open files, there's one PACB per opener. For each open of a file that is being accessed using MULTI or GMULTI access, there's one LACB per opener; for files not opened using MULTI or GMULTI there are no LACBs For each opened file, there's an FCB, which contains information common to all readers of the file (mostly a rehash of the file label, kept in memory for faster access). The FCB is pointed to both by all the file's PACBs and by the file's file label. FMAVT (File Multi-Access Vector Table) contains information on all files opened with MULTI or GMULTI access SEGMENT-ORIENTED SYSTEM TABLES Segments are broken up into two very distinct categories: data segments and code segments. Data segments are described in the Data Segment Table (DST); there's one entry for each data segment, and it contains information like the data segment length, memory address, flags, etc. That's all there's to it, a welcome change from the complicated structures in place for files, devices, and even jobs and processes. No such luck with code segments, though. The Code Segment Table (CST) contains only the entries -- also listing the segments' lengths, addresses, and other flags -- referring to all currently loaded SL segments and the program segments of the process that is currently running. Say that you want to find out about the code segments of a process given its PIN. What you want to do is to get to the entries of the CST eXtension (CSTX) table -- which is stored in a single data segment -- that correspond to the process' code segments. Your first step would be to get the "CSTX block map index", known as CSTXEIX, of the process from the process' PCB. Note that this value is listed in the PCB table layout as BLKINX and PBX. This is an index into a table called the CSTBLK, also known as the CSTXMAP. The CSTXEIXth entry in the CSTBLK contains one thing and one thing only: the address of the first CSTX entry describing the segments of the process. However, this is NOT a CSTX-relative address. This is not even a SYSGLOB-relative address. An absolute address, you say? No. This address is relative to the base of the DST, an entirely different data segment altogether! Since you can't very well MFDS CSTX entries from the DST, you have to subtract the contents of SYSGLOB location %33 from the value, and then you have the CSTX-relative index. SYSGLOB %33 (absolute %1033) is the DST-relative address of the first entry of the CSTX. Once you're done with all this -- you got the CSTXEIX from the PCB, you got the DST-relative address from the CSTBLK, and you converted it to a CSTX-relative-address -- you can now get the CSTX entries. The first entry, the one pointed to by the address you worked so hard to get, tells you how many code segments the process' has and how many users are sharing it. Subsequent entries describe each of the program's segments. In summary, the segment-oriented tables look like this: DST, one entry per data segment containing The length of the data segment The memory address of the start of the data segment Miscellaneous flags CST, one entry per SL code segment and each code segment of the currently-executing program containing The length of the code segment The memory address of the start of the code segment Miscellaneous flags CSTX, one entry per loaded program (not process! all processes running the same program share code segments) containing The number of segments in the program The number of processes running the program Also in the CSTX, one entry per segment of each loaded program, formatted like a CST entry CSTBLK (also known as CSTXMAP), one entry per program containing A DST(!)-relative pointer to the first CSTX entry belonging to the program MISCELLANEOUS SYSTEM TABLES -- SPOOLING The information on all spool files in the system is kept in the Input Device Directory (IDD) for input spool files and the Output Device Directory (ODD) for output spool files. Both of these are independent data segments, and the formats of all their entries are identical. XDD is a term used to generically talk about either the IDD or the ODD; an XDD entry is an entry that could be from the IDD or the ODD. An XDD has three kinds of entries: * One header entry, describing the table -- it contains things like the size of the table and the next input/output spool file number to be allocated. * One device header entry for each device in the system. Each entry contains the device number, device outfence (if any), and the head and tail addresses of the linked list of spool file entries belonging to that device. The 0th device header entry contains the head and tail addresses of the linked list of all spool file entries belonging not to a device, but to a device class. * One spool file entry for each existing spool file. Each entry contains the name of the spool file, the user, account, and job name that it belongs to, the disc address of its file label, etc. SPOOK's >SHOW command (especially >SHOW;@) essentially print the contents of these entries. This is a comparatively easy table to navigate. MPEX's spool-file manipulation routines -- %PURGE:SPOOL, %LISTF:SPOOL, etc. -- essentially traverse the ODD, picking up spool file file labels as they go along. Note that the addresses in the linked list are table-relative addresses of the start of the next spool file entry; 0 indicates no next entry. MISCELLANEOUS SYSTEM TABLES -- OTHERS There are several other relatively useful system tables. The LST (Loader Segment Table) contains information on all loaded program files and SL files. MPEX's %LISTF,ACCESS goes through this table to find out who may have a particular file loaded. The SIR (System Internal Resource) table contains information on all SIRs that are currently locked. If you see the system hanging up and you're afraid that it's because someone isn't releasing a SIR or you're getting a SIR deadlock, you can go into privileged mode :DEBUG and have a look at this table. The RIN (Resource Identification Number) table contains information on all RINs that are currently locked. MPEX's %LISTF,ACCESS looks at this to find out who is locking a file or waiting to lock the file. The RIT (Reply Information Table) contains information on all outstanding replies. There are other tables, having to do with logging, private volumes, message files, :WELCOME messages, you name it. I won't talk further about them simply because of space limitations. Also, most of the stand-alone tables are relatively simply structured, and you can find out how to get to them by reading the System Tables Manual and conducting some experiments with privileged mode :DEBUG. Remember that there is a brief description of every (well, almost every) table in the system and a "pointer" to its entry in the System Tables Manual in Appendix 2 of this paper. CONCLUSION System tables are not the incomprehensible monsters that they may seem to be at first glance. Once you master the various techniques described in this paper and get a reasonable familiarity with the actual contents of the tables, you should, with relatively little effort and worry, be able to access system tables just as easily as you would, say, a file or a database. Good luck and HAPPY HACKING. ACKNOWLEDGEMENTS I would like to thank the following people for reviewing and making many useful comments on this paper: Steve Cooper, Allegro Consultants Robert Green, ROBELLE Consulting Ltd. David Greer, ROBELLE Consulting Ltd. Jack Howard, Consultant, Los Angeles Stan Sieler, Allegro Consultants Jim Squires, HP Fullerton Vladimir Volokh, VESOFT, Inc. and of course everybody -- too numerous to mention -- who taught me all this stuff. May your programs never fail and your systems never crash. APPENDIX 1 -- PROGRAM 1 << Don't forget to :PREP me with ;CAP=PM! >> << Works on MPE IV and MPE V. >> $CONTROL MAIN=PROGRAM'1, NOSOURCE, USLINIT << This program: Prints out word 0 of the user's capability matrix; Prints out the job or session's job/session number; Gives the user SM capability and does a LISTUSER before (which should fail) and after. >> BEGIN INTRINSIC ASCII, COMMAND, GETPRIVMODE, GETUSERMODE, PRINT; INTEGER POINTER DL; INTEGER CAPS, DUMMY, ERROR, JS'NUMBER, LENGTH; ARRAY BUFFER'L(0:127); BYTE ARRAY BUFFER'(*)=BUFFER'L; DEFINE INDEX'PXGLOB = DL(-1) #, INDEX'PXFIXED = DL(-2) #; << Get the DL pointer >> PUSH (DL); @DL:=TOS; << Get and print word 0 of the capability matrix >> GETPRIVMODE; CAPS:=DL(-INDEX'PXGLOB+2); GETUSERMODE; MOVE BUFFER':="Capability word 0 = %"; PRINT (BUFFER'L, -21, %320); ASCII (CAPS, 8, BUFFER'); PRINT (BUFFER'L, -6, 0); << Get and print the session number >> GETPRIVMODE; JS'NUMBER:=DL(-INDEX'PXFIXED+19); GETUSERMODE; IF JS'NUMBER.(0:2)=1 THEN << JS'NUMBER.(0:2) indicates session (1) or job (2) >> MOVE BUFFER':="#S" ELSE MOVE BUFFER':="#J"; PRINT (BUFFER'L, -2, %320); LENGTH:=ASCII (JS'NUMBER.(2:14), 10, BUFFER'); PRINT (BUFFER'L, -LENGTH, 0); << First, show that LISTUSER MANAGER.SYS fails without SM >> MOVE BUFFER':="Trying LISTUSER MANAGER.SYS before getting SM"; PRINT (BUFFER'L, -45, 0); MOVE BUFFER':=("LISTUSER MANAGER.SYS", %15); COMMAND (BUFFER', ERROR, DUMMY); IF ERROR<>0 THEN BEGIN MOVE BUFFER':="CI Error # "; PRINT (BUFFER'L, -11, %320); LENGTH:=ASCII (ERROR, 10, BUFFER'); PRINT (BUFFER'L, -LENGTH, 0); END; << Now, get SM capability >> GETPRIVMODE; DL(-INDEX'PXGLOB+2).(0:1) << SM bit >> := 1; GETUSERMODE; << Now, show that LISTUSER MANAGER.SYS succeeds >> MOVE BUFFER':="Trying LISTUSER MANAGER.SYS after getting SM"; PRINT (BUFFER'L, -44, 0); MOVE BUFFER':=("LISTUSER MANAGER.SYS", %15); COMMAND (BUFFER', ERROR, DUMMY); IF ERROR<>0 THEN BEGIN MOVE BUFFER':="CI Error # "; PRINT (BUFFER'L, -11, %320); LENGTH:=ASCII (ERROR, 10, BUFFER'); PRINT (BUFFER'L, -LENGTH, 0); END; END. APPENDIX 1 -- PROGRAM 2 << Don't forget to :PREP me with ;CAP=PM! >> << Works on MPE IV; changes needed for MPE V are indicated. >> << This program assumes the existence of the DSEGREAD and MYSTACK procedures described in the text. >> $CONTROL MAIN=PROGRAM'2, NOSOURCE, USLINIT BEGIN << This program: Prints out the job/session name of the current job/session (or blanks if no job/session name.) >> INTRINSIC PRINT; EQUATE SIZE'PXGLOB = 8; << 12 for MPE V >> DEFINE PXGLOB'JIT'DSEG = PXGLOB(6).(6:10) #; << JIT data segment # >> << PXGLOB(11) for MPE V >> ARRAY PXGLOB(0:SIZE'PXGLOB-1); << Array for holding the PXGLOB data >> EQUATE OFFSET'PXGLOB = 0; << Offset of the PXGLOB in the stack dseg >> EQUATE SIZE'JIT = 61; << 67 for MPE V >> DEFINE JIT'JS'NAME = JIT(44) #; << MPE IV or MPE V >> ARRAY DSEGREAD (MYSTACK, OFFSET'PXGLOB, PXGLOB, SIZE'PXGLOB); DSEGREAD (PXGLOB'JIT'DSEG, 0, JIT, SIZE'JIT); PRINT (JIT'JS'NAME, -8, 0); END. APPENDIX 2 -- SUMMARY OF SYSTEM TABLES System tables are listed alphabetically by their initials. The chapter numbers of the tables in the System Tables Manual are given. Page numbers are not given because they may vary from release to release of each manual -- they shouldn't be too hard to find in any case. All tables are stored in memory (not on disc) unless otherwise specified. Table Chapter Table describes or contains ----- ------- --------------------------- AFT 6  One/process; part of PXFILE ASSOC 15 All :ASSOCIATE commands BKPNT 17 :DEBUG breakpoints CBT 6  Contains file system control blocks CILOG None :(CMD) USER.ACCT logons CST 2 * Loaded SL and current program's code segments CSTAB 23  Contains communications system info CSTBLK 2 * Contains pointers into the CSTX CSTX 2 All code segments in the system DCT 13 Device classes DIT 13  One/device; contains device-specific info DLT 13 Device drivers DRQ 13 * Queued disc I/O request DST 2 * Data segments FCB 6  One/file opener; file system info FLAB 6  One/file, stored on disc; file parameters FMAVT 6 Files opened with MULTI/GMULTI access ICS 13 Contains stack of interrupt-handling process IDD 14 Input spool files ILT 13 Contains interrupt info IOQ 13 * Queued I/O requests JCUT 8 * Jobs that have a CPU limit JDSD 8  One/job; job data segment info, part of JDT JDT 8  One/job; job's :FILE eqns, temp files, etc. JFEQ 8  One/job; job :FILE equation info, part of JDT JIT 8  One/job; detailed job info JJCW 8  One/job; job JCW info, part of JDT JLEQ 8  One/job; job :CLINE equation info, part of JDT JMAT 8 Jobs JPCNT 8 * Contains arcane info about jobs JTFD 8  One/job; job temp file info, part of JDT LACB 6  One/MULTI or GMULTI file opener; file system info LDT 13 LDEVs LDTX 13 LDEVs LIDTAB 17 Logging identifiers LOGBUFF 17 Contains logging buffers LOGTAB 17 Logging LPDT 13 * LDEVs; points to DITs LST 11 All loaded program and SL files MEASINFO 17 * Measurement info MVTAB 12 Mounted private volumes ODD 14 Output spool files PACB 6  One/file opener; file system info PCB 7 * Processes PCBX 7  One/process; contains PXGLOB, PXFIXED, PXFILE PJXREF None Maps PINs into job numbers PPCOM 7 MAIL messages that have been sent but not gotten PVUSER 12 Users of private volumes PXFILE 6  One/process; contains info on process' files PXFIXED 7  One/process; contains miscellaneous process info PXGLOB 7  One/process; points to JIT and JDT RIN 5 RINs (including file locks) RIT 15 Outstanding :REPLYs SBUF 13 * Contains buffers used for some system I/O's SIR 5 * Locked SIRs SRT 2 * Special memory management requests SWAPTAB 2 * Contains segment swapping info SYSGLOB 1 Contains miscellaneous system info SYSJDT 8 Contains the JDT used by system processes SYSJIT 8 Contains the JIT used by system processes TBUF 13 * Contains buffers used for terminal I/O's TDT 13  Special configurations for terminals TRL 17 * Requests for system timer (PAUSEs, timeouts, etc.) UCRQ 8 All requests to the process controller process VDD 17 Mounted labelled tapes VDSMTAB 3 * Virtual memory on discs VTAB 3 Discs XDD 14 Generic name for either the IDD or ODD * - Indicates a system table that is accessible using the SPL "INTEGER POINTER table = number;" construct. For the system table number (not necessarily equal to the data segment number), see the layout of SYSGLOB in chapter 1. If SYSGLOB cell N is indicated as containing the base of a table, then N is the table's number.  - Indicates a table that exists in MPE V but not in MPE IV.  - Indicates a table documented in the MPE V manual but not in the MPE IV manual.  - Indicates that there is more than one of these tables, one for each one of a number of objects. For instance, a JIT, of which there's one per job, and an FLAB, of which there's one per file. All tables not so marked are unique.
Go to Adager's index of technical papers