[HECnet] Tops-20 Disk Quotas (was Anonymous FAL (Tops-20))

[Johnny Billquist]

Unix on PDP-7 is 1969. On a PDP-11, more or less 1970(ish).
But I admit that I didn't fully think about TENEX, which I guess 
actually makes them about the same age.

And yeah, mkdir is simpler. Creating a subdirectory in TOPS-20 is more 
complicated, but it isn't hell. :-)

And everyone is always screaming for more disk, memory and CPU. Nothing 
new there...

Also, yes, emulation makes it rather nice to run these old systems. 
Plenty of resources available (from their perspective).

I have about 8GB of stuff on RSX here. Talk about wasting disk. :-)
And that is by just using two disk drives... Such disks didn't even 
exist back then. And my network screams. ftp transfer from RSX at about 
1.5MB/s, which is way more than the bandwidth of a Unibus, or even good 
old 10Mbit/s ethernet.
(Not to mention I have a multiprocessor PDP-11/74...)

Those are the moments when I really appreciate emulations...

   Johnny


On 2019-07-07 03:15, Thomas DeBellis wrote:
> Yes, perhaps I wrote that poorly; my apologies.  Accounts and 
> directories are quite clearly separated; you can run Tops-20 without 
> _any_ accounts whatsoever.  PANDA does this as does the standard DEC 
> distribution.  We (Columbia) didn't because we had to do charge back and 
> sold time and you can use the account functionality to do some nifty 
> things, like set your scheduler class.
> 
> We also had a RSTS/E System, running on a PDP-11/70 (possibly one of the 
> finest computers /ever///).  I don't remember what we were using for 
> DASD, but it wasn't RM03.  That was an white platter disk that I 
> remember remember mounting when we switching between RT-11, RSX and a 
> very early version of Unix on a PDP-11/40 in the CS lap at WPI. I think 
> maybe Columbia had something larger like an RP04.  Anyway, I remember it 
> being separate, whereas the RM03 is in the tower. We sadly unplugged the 
> RSTS system when we got our fourth 20.  The only Basic that I ever saw 
> that had RSTS muscle was on the DTSS.
> 
> Doing sub-dirs on the 20 /is/ beautiful and there are features that I 
> appreciate today over everything else I've seen.  The grammar was 
> extremely well thought out.  But under the covers, if you had 20 to 30 
> thousand users ids to run after and frequent turn over, you had to write 
> custom software to do the group management and id creation.  I know, I 
> wrote some of it. BUILD /is /dandy, but consider all the confusing 
> options you have to do get the access right, viz:
> 
>   ABORT DEFAULT-FILE-PROTECTION DIRECTORY-GROUP FILES-ONLY
> GENERATIONS KILL       LIST MAXIMUM-SUBDIRECTORIES
>   PASSWORD    PERMANENT PROTECTION              PUSH
>   SECURESUBDIRECTORY-USER-GROUPTOPS10-PROJECT-PROGRAMMER-NUMBER
> USER-OF-GROUP WORKING
> 
> See that PUSH command?  That's so you can go recursive when (not if) 
> something breaks and come back and try it again.  Groups are far more 
> powerful than Unix's laughable excuse, but they are not straightforward 
> to implement as clash is not a bug, but rather a feature.  And you can't 
> just have a user of a group unless it's allowed in the sub-directory 
> user group.  Get any of that wrong and you just created a sub-directory 
> that the user can't use and they're ... not happy ...
> 
> The quotas are just plain tedious because unless you set the magic bit, 
> you have to grab it from the superior and then guess how much (which is 
> never right) or ask the user (who has no idea or wants everything).  And 
> then you have to explain why SECURE isn't necessary...
> 
> And then there are all these other goofy things that they should have 
> just ditched and put into in ^ECREATE so your phone doesn't ring.
> 
>   ABSOLUTE-INTERNET-SOCKETS ACCOUNT-DEFAULT   ADMINISTRATOR 
> ARCHIVE-ONLINE-EXPIRED-FILES
>   CHARGE-LIMITED     CONFIDENTIAL      DECNET-ACCESS DISABLE
>   ENABLE     ENQ-DEQ           EXPIRATION-OF-PASSWORDEXPIRE
>   FROZEN                     INTERNET-ACCESS   INTERNET-WIZARD     IPCF
>   MAINTENANCE MUST-RUN-PROGRAM  NUMBEROFFLINE-EXPIRATION-DEFAULT
>   ONLINE-EXPIRATION-DEFAULTOPERATOR PRESERVE               
> REPEAT-LOGIN-MESSAGES
>   SEMI-OPERATOR WHEEL
> 
> This is visible list and t it guarantees your phone rings because if 
> they try the BUILD and of them, it will break.  As a matter of fact, 
> except for a very limited subset (which does not include creating 
> sub-directories), it is going to break.  So that's fine if you feel like 
> chatting, but it's almost never a short call.
> 
>     Why isn't secure /secure/?
>          Because our ACJ doesn't need enable the hooks.
>     Why?
>          Because we don't need them.
>     Well, shouldn't */I/* be secure?
>          Yes, you should be... I mean, you are.  (he sighs)
> 
> And, my favorite:
> 
>     What's a WHEEL??
>          "Blessed are they who run around in circles" (he begins intoning)
>     /??/
>          "For they shall be known as Wheels" (he finishes intoning)
>     /????/
>          It means you have complete unfettered and limited system
>     access.  Beyond root or administrator.
>     Oh!!  Well I should have that.
>          Indeed?  Why?
>     Because my: (pick one)
> 
>      1. Thesis Advisor
>      2. Dean
>      3. Manager
>      4. Mother
>      5. Spiritual Advisor
>      6. Boyfriend
>      7. Dog
> 
>     thinks I should...
> 
>          And yet I remain unconvinced. However, don't let that stop them
>     from hiring you.
> 
> Now, let's compare that whole saga with the effortlessness of mkdir or 
> md--boom you're done and no phone call.  Of course you have a point that 
> BUILD isn't *that* hard in theory.  However, in practice as compared 
> with the former two, was a serious pain in the ass and I think it 
> annoying in this day and age.  It was that complicated because it had to 
> be because of the huge user populations.
> 
> Yes, we had bunches of disks, too; we had at least one RP07, a number of 
> RA81's on an HSC50 (clustered) and I believe something on the order of 
> 20 RP06's (I'd have to look at my copy of the machine room diagram), 8 
> tape drives to back everything up and printers and ...  Remember that 
> population?  It still wasn't enough.  It's one thing to write a small 
> program for an introductory class, but when you really start getting on 
> it; writing papers, simulators or compilers.  You just swallow disk 
> space and that's before you even talk about anything remotely 
> approaching multimedia, which was unthinkable.
> 
> Perhaps the following example is illustrative: In order to validate my 
> FTP server, I needed some 'decent' sized data sets--things I could look 
> at and immediately notice any obvious problems.  So I downloaded some of 
> my favorite Sherlock Holmes and Oscar Wilde novels from Project 
> Gutenberg along with some other goodies.  All told, over 9,000 pages.
> 
> Now, let's suppose you wanted to do a longitudinal textual analysis of 
> stylistic changes in Abraham Lincoln's speeches.  It is instructive to 
> compare the first and second inaugural addresses using the Gettysburg 
> address as a linking document. Well, that's 12 pages right there before 
> you've written anything, over 10% of your quota.  We were always 
> screaming for more disk.  And CPU.  And memory.
> 
> My KLH10 is over 200 times faster than a KL and I have 5 RP07's, with 
> one piggy user (me), two medium users (my wife and brother) and some 
> assorted guests.  If you compare that with a KL10B with 70 signed on and 
> 20,000 trying to sign on, you can see why those disk drives simply 
> weren't enough.  Nothing but a 3850 would have been and we weren't 
> allowed to use that.  We have one holding the 1980 census, the 
> equivalent of 4,720 RP06's (on the order of a terabyte).  There was a 
> lot of drooling on the floor, but it was dedicated to research.
> 
> I wasn't aware that Unix was older than Tops-20.   What basis do you 
> have for making this statement?  I had though it younger.  The initial 
> Unix release date is November 3, 1971 whereas TENEX came on the air in 
> June 15, 1970, more than a year beforehand.  Unfortunately, I don't have 
> my Bell System Technical Journals handy (still in boxes), so I don't 
> immediately recall the period between Bell pulling the plug on Multics 
> and Thompson began playing with that cast off PDP-7. Clearly however, 
> BBN was working on TENEX in the late 1960's and sold the page box as a 
> commercial product.
> 
>> ------------------------------------------------------------------------
>> On 7/5/2019 3:40 PM, Johnny Billquist wrote:
>>
>> Well, accounts and directories are not clearly separated things under 
>> TOPS-20, as you yourself noted.
>> And disk quotas were annoying to deal with.
>>
>> Back around the same time, I was using RSTS/E at school, and there you 
>> had disk quotas too, and no subdirectories. Also, the default quota 
>> was 20 blocks (10 Kbyte). Felt quite acceptable at the time. One RM03 
>> for four schools sharing one PDP-11/70.
>>
>> But creating directories under TOPS-20 was not *that* hard. There was 
>> the BUILD command, which sorted out most things rather simply. But I 
>> don't know how you'd do it programmatically.
>>
>> The fact that you could have sub-users on the other hand was one of 
>> the most beautiful things of TOPS-20. And it implicitly already gave 
>> you groups. At university, each course there was a user, and all 
>> students were sub-user to that, belonging thus to the same group. 
>> Managed by the teacher, who had the parent account.
>>
>> As for maximum disk, well... You could have RP07 disks. At half a gig, 
>> that was pretty decent. Each of our -2060 had one RP07. And one had 
>> one RP06, while the other one had three RP06 drives.
>>
>> And Unix is older than TOPS-20, and ran on more limited resources, and 
>> still handled subdirectories and quotas cleaner. So I don't think it's 
>> fair to just blame old age or limited resources. A PDP-10 had vast 
>> resources compared to many other things...
>>
>>   Johnny
>>
>> On 2019-07-05 20:57, Thomas DeBellis wrote:
>>> Oh, it's something beyond annoying, but it's not the accounting 
>>> system confounding you; that can be completely disabled (I have it 
>>> off on my systems).  The policy is actually built into the Tops-20 
>>> file system itself.
>>>
>>> Directories under Tops-20 are vastly different--both in concept and 
>>> implementation--from anything else that I've seen (and I did a lot of 
>>> research into file system design at one particular job). Directory 
>>> creation is cumbersome, typically requiring expert level intervention 
>>> or significant programming.  However, it's whaay better than what 
>>> Tops-10 had at the time (nothing), ITS (don't ask), WAITS (nothing) 
>>> or MVS (partitioned data sets, a true hack).
>>>
>>> Create a directory under Unix? mkdir.  Easyn  peasy. Windows? md, 
>>> unless you are running quotas.  Also no heavy lift.
>>>
>>> Tops-20 got more and more complex.  In addition to having to take 
>>> quota away from the superior and hand it over to the sub-directory, 
>>> unless you are running PANDA modifications, you have to create an 
>>> access group and allocate it or the poor user can't see his own 
>>> sub-directory.   Group management can be confusing if you are running 
>>> super-domestic structures and downright tedious for regular 
>>> structures, otherwise.  There was more; Yeesh...   Instead of trying 
>>> to check for every possible problem beforehand, it was sometimes 
>>> easier to catch errors from the CRDIR%, go recursive and modify the 
>>> superior (and on up).
>>>
>>> You can defeat some of this.  Setting CD%NSQ will cause CRDIR% to no 
>>> update the the superior, but you need rights to do it.  I always 
>>> thought that there was a better way to do this, perhaps with an IPCF% 
>>> based client/server application, coupled with some changes to the 
>>> access control job.
>>>
>>> Why all this hair?  Directories were considered precious resources.  
>>> Why would that be?  Consider what happens when you try to fit (or 
>>> cram) a user population of over 25,000 students onto the triple 180 
>>> MB disk structures of the time (the maximum you could do in 1980's).  
>>> You get measly user permanent quotas of 100 pages (250KB), working of 
>>> 1,000.  Not much.
>>>
>>> It's a vastly different world now.  So Tops-20 needs a mkdir, but 
>>> that would need to talk to a privileged backend with policy and 
>>> directory creation smarts.  I think that would be pretty friendly; 
>>> definitely easier than trying to suss out BUILD or ^ECREATE.
>>>> ------------------------------------------------------------------------ 
>>>>
>>>> On 7/4/2019 2:48 PM, Johnny Billquist wrote:
>>>>
>>>> The one annoying detail of the account system in TOPS-20 is that 
>>>> user disk quotas are on a per directory basis. So you have to 
>>>> manually move your disk quota around for your subdirectories.
>>>>> ------------------------------------------------------------------------ 
>>>>>
>>>>> On 2019-07-04 04:01, Thomas DeBellis wrote:
>>>>>
>>>>> Tops-20 is vastly different from Unix (and I believe also VMS) as 
>>>>> to how it manages user ids and accounts.  Parts of the 
>>>>> authentication paradigm are very tightly woven into the the file 
>>>>> system.  Briefly,
>>>>>
>>>>>   * A user id is a login-able directory (I.E., one that doesn't have
>>>>>     apassword and is not set FILES-ONLY).  In addition to basic OS
>>>>>     restrictions which prevent you from viewing file system meta-data
>>>>>     unless you have appropriate authorization, an access control job
>>>>>     (ACJ) is layered on top of this which can even restrict
>>>>>     privileged users.
>>>>>   * Accounts are either validated out of a binary accounting file in
>>>>>     monitor space (which is compiled from ASCII source) or via the
>>>>>     ACJ.     Accounts can have multiple users or systems processes
>>>>>     (such as spoolers) creating billing records. Users can switch
>>>>>     between accounts on a per-job, per-fork and intra-program basis
>>>>>     (a program can decide to bill certain portions of its activity to
>>>>>     different accounts).
>>>>>   * The obvious benefit is that there is no password file to attack
>>>>>     or steal and you can't even tell that there is an accounting
>>>>>     file; probing passwords is monitored and a certain amount of
>>>>>     intervention is done.  It is /extremely/ fast. No /etc/passwd to
>>>>>     grovel.
>>>>>
>>>>> However, a deleterious side-effect is that once an id is created, 
>>>>> it can be used for _anything_, including online interactive login.
>>>>>
>>>>> On a PANDA monitor, is possible to specify a user id as FTP-ONLY, 
>>>>> but neither the supplied 5 series ACJ nor the EXEC do anything with 
>>>>> it.  Historically, the Tops-20 FTP server implemented ANONYMOUS 
>>>>> usage by parsing for the login user atom ANONYMOUS and then 
>>>>> swallowing anything for the password (what was typically supplied 
>>>>> was an email addresses). This was then hardwired into a local id.
>>>>>
>>>>> Artifacts of this still exist in certain browers.  Guess who 
>>>>> supplies IEUSER@ as the email address password for ANONYOUS usage?
>>>>>
>>>>> I recall that this is the approach that we had to use with Tops-20 
>>>>> FAL.  The Extended  Mode FTP server that I wrote is configurable 
>>>>> via a file to specify the underlying id and password.  More 
>>>>> productization would probably including having the ACJ enforce 
>>>>> FTP-ONLY on LOGIN% or CRJOB% and having the EXEC parse for and 
>>>>> display FTP-ONLY.  Probably about two weeks' part time work as I 
>>>>> recall.  Might have to consider Batch policy.
>>>>>
>>>>> One approach here could be to lift the ANONYMOUS code out of EFTPSR 
>>>>> and drop it into FAL and then do the changes to the ACJ and EXEC. 
>>>>> I'm just surprised none of the HECnet Tops-10 or Tops-20 nerds have 
>>>>> done it (there is some commonality in some of the sources).
>>>>>
>>>>> Since Tops-20 has a BLISS compiler which implements BLISS COMMON 
>>>>> (my first training at DEC as an employee was to write code that 
>>>>> would cross compile under VMS, RSX, Tops-10 and Tops-20).  I think 
>>>>> it might be useful to review some of the VMS DECnet source, if any 
>>>>> of that is available.  It might be possible to lift some 
>>>>> functionality, which could be fun.
>>>>>
>>>>> Does the VMS hobbiest license get you source code?
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>> On 7/3/2019 7:21 PM, Johnny Billquist wrote:
>>>>>>
>>>>>> VMS, as someone else mentioned, have a default account for FAL.
>>>>>>
>>>>>> RSX does not have that.  However, you can use proxy access in RSX 
>>>>>> to achieve something similar.  Enable incoming and outgoing proxy, 
>>>>>> and define a default account that incoming requests should be 
>>>>>> using that way.
>>>>>>
>>>>>> If TOPS-20 can do this I don't know.  But it's a suggestion for 
>>>>>> something else/more to check.
>>>>>>
>>>>>>   Johnny
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>> On 2019-07-03 14:15, Thomas DeBellis wrote:
>>>>>>>
>>>>>>> I have some software that I'd like to post, but don't recall how 
>>>>>>> to configure FAL to allow for an anonymous connection; to 
>>>>>>> download from a restricted directory.
>>>>>>>
>>>>>>> I know how to do it for the FTP server (seeing as I wrote it), 
>>>>>>> but ... different code base.
>>>>>>>
>>>>>>> I can only vaguely remember what we did for CCnet at Columbia 
>>>>>>> University in the 1980's, but I think it was kind of a hack.
>>
>>

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol