[HECnet] Tops-20 Disk Quotas (was Anonymous FAL (Tops-20))

[Thomas DeBellis]

Yes, and it would probably be interesting to compare Multics' directory 
creation paradigm, what with it's B2 rating and everything.

Unix, Windows and OS/2 are just plain easy peasy lemon squeezy. ITS and 
MVS are straightforward, if arcane.

I can't remember RSX or VMS.

Tops-20 just really wants a candy wrapper to cover up all that nasty hair.

On 7/6/2019 10:29 PM, Johnny Billquist wrote:
> (And you could of course trace Unix back into Multics as well...)
>
>   Johnny
>
> On 2019-07-07 04:06, Johnny Billquist wrote:
>> Unix on PDP-7 is 1969. On a PDP-11, more or less 1970(ish).
>> But I admit that I didn't fully think about TENEX, which I guess 
>> actually makes them about the same age.
>>
>> And yeah, mkdir is simpler. Creating a subdirectory in TOPS-20 is 
>> more complicated, but it isn't hell. :-)
>>
>> And everyone is always screaming for more disk, memory and CPU. 
>> Nothing new there...
>>
>> Also, yes, emulation makes it rather nice to run these old systems. 
>> Plenty of resources available (from their perspective).
>>
>> I have about 8GB of stuff on RSX here. Talk about wasting disk. :-)
>> And that is by just using two disk drives... Such disks didn't even 
>> exist back then. And my network screams. ftp transfer from RSX at 
>> about 1.5MB/s, which is way more than the bandwidth of a Unibus, or 
>> even good old 10Mbit/s ethernet.
>> (Not to mention I have a multiprocessor PDP-11/74...)
>>
>> Those are the moments when I really appreciate emulations...
>>
>>    Johnny
>>
>>
>> On 2019-07-07 03:15, Thomas DeBellis wrote:
>>> Yes, perhaps I wrote that poorly; my apologies.  Accounts and 
>>> directories are quite clearly separated; you can run Tops-20 without 
>>> _any_ accounts whatsoever.  PANDA does this as does the standard DEC 
>>> distribution.  We (Columbia) didn't because we had to do charge back 
>>> and sold time and you can use the account functionality to do some 
>>> nifty things, like set your scheduler class.
>>>
>>> We also had a RSTS/E System, running on a PDP-11/70 (possibly one of 
>>> the finest computers /ever///).  I don't remember what we were using 
>>> for DASD, but it wasn't RM03.  That was an white platter disk that I 
>>> remember remember mounting when we switching between RT-11, RSX and 
>>> a very early version of Unix on a PDP-11/40 in the CS lap at WPI. I 
>>> think maybe Columbia had something larger like an RP04.  Anyway, I 
>>> remember it being separate, whereas the RM03 is in the tower. We 
>>> sadly unplugged the RSTS system when we got our fourth 20.  The only 
>>> Basic that I ever saw that had RSTS muscle was on the DTSS.
>>>
>>> Doing sub-dirs on the 20 /is/ beautiful and there are features that 
>>> I appreciate today over everything else I've seen.  The grammar was 
>>> extremely well thought out.  But under the covers, if you had 20 to 
>>> 30 thousand users ids to run after and frequent turn over, you had 
>>> to write custom software to do the group management and id 
>>> creation.  I know, I wrote some of it. BUILD /is /dandy, but 
>>> consider all the confusing options you have to do get the access 
>>> right, viz:
>>>
>>>   ABORT DEFAULT-FILE-PROTECTION DIRECTORY-GROUP FILES-ONLY
>>> GENERATIONS KILL       LIST MAXIMUM-SUBDIRECTORIES
>>>   PASSWORD    PERMANENT PROTECTION              PUSH
>>>   SECURESUBDIRECTORY-USER-GROUPTOPS10-PROJECT-PROGRAMMER-NUMBER
>>> USER-OF-GROUP WORKING
>>>
>>> See that PUSH command?  That's so you can go recursive when (not if) 
>>> something breaks and come back and try it again. Groups are far more 
>>> powerful than Unix's laughable excuse, but they are not 
>>> straightforward to implement as clash is not a bug, but rather a 
>>> feature.  And you can't just have a user of a group unless it's 
>>> allowed in the sub-directory user group. Get any of that wrong and 
>>> you just created a sub-directory that the user can't use and they're 
>>> ... not happy ...
>>>
>>> The quotas are just plain tedious because unless you set the magic 
>>> bit, you have to grab it from the superior and then guess how much 
>>> (which is never right) or ask the user (who has no idea or wants 
>>> everything).  And then you have to explain why SECURE isn't 
>>> necessary...
>>>
>>> And then there are all these other goofy things that they should 
>>> have just ditched and put into in ^ECREATE so your phone doesn't ring.
>>>
>>>   ABSOLUTE-INTERNET-SOCKETS ACCOUNT-DEFAULT   ADMINISTRATOR 
>>> ARCHIVE-ONLINE-EXPIRED-FILES
>>>   CHARGE-LIMITED     CONFIDENTIAL      DECNET-ACCESS DISABLE
>>>   ENABLE     ENQ-DEQ           EXPIRATION-OF-PASSWORDEXPIRE
>>>   FROZEN                     INTERNET-ACCESS INTERNET-WIZARD     IPCF
>>>   MAINTENANCE MUST-RUN-PROGRAM NUMBEROFFLINE-EXPIRATION-DEFAULT
>>>   ONLINE-EXPIRATION-DEFAULTOPERATOR PRESERVE REPEAT-LOGIN-MESSAGES
>>>   SEMI-OPERATOR WHEEL
>>>
>>> This is visible list and t it guarantees your phone rings because if 
>>> they try the BUILD and of them, it will break.  As a matter of fact, 
>>> except for a very limited subset (which does not include creating 
>>> sub-directories), it is going to break. So that's fine if you feel 
>>> like chatting, but it's almost never a short call.
>>>
>>>     Why isn't secure /secure/?
>>>          Because our ACJ doesn't need enable the hooks.
>>>     Why?
>>>          Because we don't need them.
>>>     Well, shouldn't */I/* be secure?
>>>          Yes, you should be... I mean, you are.  (he sighs)
>>>
>>> And, my favorite:
>>>
>>>     What's a WHEEL??
>>>          "Blessed are they who run around in circles" (he begins 
>>> intoning)
>>>     /??/
>>>          "For they shall be known as Wheels" (he finishes intoning)
>>>     /????/
>>>          It means you have complete unfettered and limited system
>>>     access.  Beyond root or administrator.
>>>     Oh!!  Well I should have that.
>>>          Indeed?  Why?
>>>     Because my: (pick one)
>>>
>>>      1. Thesis Advisor
>>>      2. Dean
>>>      3. Manager
>>>      4. Mother
>>>      5. Spiritual Advisor
>>>      6. Boyfriend
>>>      7. Dog
>>>
>>>     thinks I should...
>>>
>>>          And yet I remain unconvinced. However, don't let that stop 
>>> them
>>>     from hiring you.
>>>
>>> Now, let's compare that whole saga with the effortlessness of mkdir 
>>> or md--boom you're done and no phone call.  Of course you have a 
>>> point that BUILD isn't *that* hard in theory. However, in practice 
>>> as compared with the former two, was a serious pain in the ass and I 
>>> think it annoying in this day and age.  It was that complicated 
>>> because it had to be because of the huge user populations.
>>>
>>> Yes, we had bunches of disks, too; we had at least one RP07, a 
>>> number of RA81's on an HSC50 (clustered) and I believe something on 
>>> the order of 20 RP06's (I'd have to look at my copy of the machine 
>>> room diagram), 8 tape drives to back everything up and printers and 
>>> ...  Remember that population? It still wasn't enough.  It's one 
>>> thing to write a small program for an introductory class, but when 
>>> you really start getting on it; writing papers, simulators or 
>>> compilers.  You just swallow disk space and that's before you even 
>>> talk about anything remotely approaching multimedia, which was 
>>> unthinkable.
>>>
>>> Perhaps the following example is illustrative: In order to validate 
>>> my FTP server, I needed some 'decent' sized data sets--things I 
>>> could look at and immediately notice any obvious problems.  So I 
>>> downloaded some of my favorite Sherlock Holmes and Oscar Wilde 
>>> novels from Project Gutenberg along with some other goodies.  All 
>>> told, over 9,000 pages.
>>>
>>> Now, let's suppose you wanted to do a longitudinal textual analysis 
>>> of stylistic changes in Abraham Lincoln's speeches. It is 
>>> instructive to compare the first and second inaugural addresses 
>>> using the Gettysburg address as a linking document. Well, that's 12 
>>> pages right there before you've written anything, over 10% of your 
>>> quota.  We were always screaming for more disk.  And CPU.  And memory.
>>>
>>> My KLH10 is over 200 times faster than a KL and I have 5 RP07's, 
>>> with one piggy user (me), two medium users (my wife and brother) and 
>>> some assorted guests.  If you compare that with a KL10B with 70 
>>> signed on and 20,000 trying to sign on, you can see why those disk 
>>> drives simply weren't enough. Nothing but a 3850 would have been and 
>>> we weren't allowed to use that.  We have one holding the 1980 
>>> census, the equivalent of 4,720 RP06's (on the order of a 
>>> terabyte).  There was a lot of drooling on the floor, but it was 
>>> dedicated to research.
>>>
>>> I wasn't aware that Unix was older than Tops-20.   What basis do you 
>>> have for making this statement?  I had though it younger.  The 
>>> initial Unix release date is November 3, 1971 whereas TENEX came on 
>>> the air in June 15, 1970, more than a year beforehand.  
>>> Unfortunately, I don't have my Bell System Technical Journals handy 
>>> (still in boxes), so I don't immediately recall the period between 
>>> Bell pulling the plug on Multics and Thompson began playing with 
>>> that cast off PDP-7. Clearly however, BBN was working on TENEX in 
>>> the late 1960's and sold the page box as a commercial product.
>>>
>>>> ------------------------------------------------------------------------ 
>>>>
>>>> On 7/5/2019 3:40 PM, Johnny Billquist wrote:
>>>>
>>>> Well, accounts and directories are not clearly separated things 
>>>> under TOPS-20, as you yourself noted.
>>>> And disk quotas were annoying to deal with.
>>>>
>>>> Back around the same time, I was using RSTS/E at school, and there 
>>>> you had disk quotas too, and no subdirectories. Also, the default 
>>>> quota was 20 blocks (10 Kbyte). Felt quite acceptable at the time. 
>>>> One RM03 for four schools sharing one PDP-11/70.
>>>>
>>>> But creating directories under TOPS-20 was not *that* hard. There 
>>>> was the BUILD command, which sorted out most things rather simply. 
>>>> But I don't know how you'd do it programmatically.
>>>>
>>>> The fact that you could have sub-users on the other hand was one of 
>>>> the most beautiful things of TOPS-20. And it implicitly already 
>>>> gave you groups. At university, each course there was a user, and 
>>>> all students were sub-user to that, belonging thus to the same 
>>>> group. Managed by the teacher, who had the parent account.
>>>>
>>>> As for maximum disk, well... You could have RP07 disks. At half a 
>>>> gig, that was pretty decent. Each of our -2060 had one RP07. And 
>>>> one had one RP06, while the other one had three RP06 drives.
>>>>
>>>> And Unix is older than TOPS-20, and ran on more limited resources, 
>>>> and still handled subdirectories and quotas cleaner. So I don't 
>>>> think it's fair to just blame old age or limited resources. A 
>>>> PDP-10 had vast resources compared to many other things...
>>>>
>>>>   Johnny
>>>>
>>>> On 2019-07-05 20:57, Thomas DeBellis wrote:
>>>>> Oh, it's something beyond annoying, but it's not the accounting 
>>>>> system confounding you; that can be completely disabled (I have it 
>>>>> off on my systems). The policy is actually built into the Tops-20 
>>>>> file system itself.
>>>>>
>>>>> Directories under Tops-20 are vastly different--both in concept 
>>>>> and implementation--from anything else that I've seen (and I did a 
>>>>> lot of research into file system design at one particular job). 
>>>>> Directory creation is cumbersome, typically requiring expert level 
>>>>> intervention or significant programming.  However, it's whaay 
>>>>> better than what Tops-10 had at the time (nothing), ITS (don't 
>>>>> ask), WAITS (nothing) or MVS (partitioned data sets, a true hack).
>>>>>
>>>>> Create a directory under Unix? mkdir.  Easyn  peasy. Windows? md, 
>>>>> unless you are running quotas.  Also no heavy lift.
>>>>>
>>>>> Tops-20 got more and more complex.  In addition to having to take 
>>>>> quota away from the superior and hand it over to the 
>>>>> sub-directory, unless you are running PANDA modifications, you 
>>>>> have to create an access group and allocate it or the poor user 
>>>>> can't see his own sub-directory.   Group management can be 
>>>>> confusing if you are running super-domestic structures and 
>>>>> downright tedious for regular structures, otherwise.  There was 
>>>>> more; Yeesh...   Instead of trying to check for every possible 
>>>>> problem beforehand, it was sometimes easier to catch errors from 
>>>>> the CRDIR%, go recursive and modify the superior (and on up).
>>>>>
>>>>> You can defeat some of this.  Setting CD%NSQ will cause CRDIR% to 
>>>>> no update the the superior, but you need rights to do it.  I 
>>>>> always thought that there was a better way to do this, perhaps 
>>>>> with an IPCF% based client/server application, coupled with some 
>>>>> changes to the access control job.
>>>>>
>>>>> Why all this hair?  Directories were considered precious 
>>>>> resources. Why would that be?  Consider what happens when you try 
>>>>> to fit (or cram) a user population of over 25,000 students onto 
>>>>> the triple 180 MB disk structures of the time (the maximum you 
>>>>> could do in 1980's). You get measly user permanent quotas of 100 
>>>>> pages (250KB), working of 1,000.  Not much.
>>>>>
>>>>> It's a vastly different world now.  So Tops-20 needs a mkdir, but 
>>>>> that would need to talk to a privileged backend with policy and 
>>>>> directory creation smarts.  I think that would be pretty friendly; 
>>>>> definitely easier than trying to suss out BUILD or ^ECREATE.
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>> On 7/4/2019 2:48 PM, Johnny Billquist wrote:
>>>>>>
>>>>>> The one annoying detail of the account system in TOPS-20 is that 
>>>>>> user disk quotas are on a per directory basis. So you have to 
>>>>>> manually move your disk quota around for your subdirectories.
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>> On 2019-07-04 04:01, Thomas DeBellis wrote:
>>>>>>>
>>>>>>> Tops-20 is vastly different from Unix (and I believe also VMS) 
>>>>>>> as to how it manages user ids and accounts. Parts of the 
>>>>>>> authentication paradigm are very tightly woven into the the file 
>>>>>>> system.  Briefly,
>>>>>>>
>>>>>>>   * A user id is a login-able directory (I.E., one that doesn't 
>>>>>>> have
>>>>>>>     apassword and is not set FILES-ONLY).  In addition to basic OS
>>>>>>>     restrictions which prevent you from viewing file system 
>>>>>>> meta-data
>>>>>>>     unless you have appropriate authorization, an access control 
>>>>>>> job
>>>>>>>     (ACJ) is layered on top of this which can even restrict
>>>>>>>     privileged users.
>>>>>>>   * Accounts are either validated out of a binary accounting 
>>>>>>> file in
>>>>>>>     monitor space (which is compiled from ASCII source) or via the
>>>>>>>     ACJ.     Accounts can have multiple users or systems processes
>>>>>>>     (such as spoolers) creating billing records. Users can switch
>>>>>>>     between accounts on a per-job, per-fork and intra-program basis
>>>>>>>     (a program can decide to bill certain portions of its 
>>>>>>> activity to
>>>>>>>     different accounts).
>>>>>>>   * The obvious benefit is that there is no password file to attack
>>>>>>>     or steal and you can't even tell that there is an accounting
>>>>>>>     file; probing passwords is monitored and a certain amount of
>>>>>>>     intervention is done.  It is /extremely/ fast. No 
>>>>>>> /etc/passwd to
>>>>>>>     grovel.
>>>>>>>
>>>>>>> However, a deleterious side-effect is that once an id is 
>>>>>>> created, it can be used for _anything_, including online 
>>>>>>> interactive login.
>>>>>>>
>>>>>>> On a PANDA monitor, is possible to specify a user id as 
>>>>>>> FTP-ONLY, but neither the supplied 5 series ACJ nor the EXEC do 
>>>>>>> anything with it.  Historically, the Tops-20 FTP server 
>>>>>>> implemented ANONYMOUS usage by parsing for the login user atom 
>>>>>>> ANONYMOUS and then swallowing anything for the password (what 
>>>>>>> was typically supplied was an email addresses). This was then 
>>>>>>> hardwired into a local id.
>>>>>>>
>>>>>>> Artifacts of this still exist in certain browers. Guess who 
>>>>>>> supplies IEUSER@ as the email address password for ANONYOUS usage?
>>>>>>>
>>>>>>> I recall that this is the approach that we had to use with 
>>>>>>> Tops-20 FAL.  The Extended  Mode FTP server that I wrote is 
>>>>>>> configurable via a file to specify the underlying id and 
>>>>>>> password.  More productization would probably including having 
>>>>>>> the ACJ enforce FTP-ONLY on LOGIN% or CRJOB% and having the EXEC 
>>>>>>> parse for and display FTP-ONLY.  Probably about two weeks' part 
>>>>>>> time work as I recall.  Might have to consider Batch policy.
>>>>>>>
>>>>>>> One approach here could be to lift the ANONYMOUS code out of 
>>>>>>> EFTPSR and drop it into FAL and then do the changes to the ACJ 
>>>>>>> and EXEC. I'm just surprised none of the HECnet Tops-10 or 
>>>>>>> Tops-20 nerds have done it (there is some commonality in some of 
>>>>>>> the sources).
>>>>>>>
>>>>>>> Since Tops-20 has a BLISS compiler which implements BLISS COMMON 
>>>>>>> (my first training at DEC as an employee was to write code that 
>>>>>>> would cross compile under VMS, RSX, Tops-10 and Tops-20).  I 
>>>>>>> think it might be useful to review some of the VMS DECnet 
>>>>>>> source, if any of that is available.  It might be possible to 
>>>>>>> lift some functionality, which could be fun.
>>>>>>>
>>>>>>> Does the VMS hobbiest license get you source code?
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> On 7/3/2019 7:21 PM, Johnny Billquist wrote:
>>>>>>>>
>>>>>>>> VMS, as someone else mentioned, have a default account for FAL.
>>>>>>>>
>>>>>>>> RSX does not have that.  However, you can use proxy access in 
>>>>>>>> RSX to achieve something similar.  Enable incoming and outgoing 
>>>>>>>> proxy, and define a default account that incoming requests 
>>>>>>>> should be using that way.
>>>>>>>>
>>>>>>>> If TOPS-20 can do this I don't know.  But it's a suggestion for 
>>>>>>>> something else/more to check.
>>>>>>>>
>>>>>>>>   Johnny
>>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>>
>>>>>>>>> On 2019-07-03 14:15, Thomas DeBellis wrote:
>>>>>>>>>
>>>>>>>>> I have some software that I'd like to post, but don't recall 
>>>>>>>>> how to configure FAL to allow for an anonymous connection; to 
>>>>>>>>> download from a restricted directory.
>>>>>>>>>
>>>>>>>>> I know how to do it for the FTP server (seeing as I wrote it), 
>>>>>>>>> but ... different code base.
>>>>>>>>>
>>>>>>>>> I can only vaguely remember what we did for CCnet at Columbia 
>>>>>>>>> University in the 1980's, but I think it was kind of a hack.
>>>>
>>>>
>>
>