The Memory Protection Debacle

The next posts are taken from here.

I wrote them a while back.

=============== post #70
Hi Everyone,


I'm going to speak without knowing what I'm talking about, using only preconceived notions on this subject. (NOTHING new there, huh?)


The way I see things.


AOS worked, AND multi-tasked WITHOUT memory protection, SO, just what is it that we get WITH memory protection besides a whole whack of non-working SW?


Now, to the preconceived notions.

1) I think that memory protection adds to the OS, as in, slows it down.
2) I don't think ANY SW HAS to have memory protection, that we know of. (This is of course, IF the SW writer KNOWS EXACTLY what he is doing.)
3) I think that it is in fact possible to write SW WITHOUT memory protection that can do things that CAN'T be done WITH memory protection in an OS. IOW, I think some unique applications ARE POSSIBLE in what may be the LAST no memory protection OS left.

Can anyone say I'm wrong on all three counts? (I can't think of anything else, but it would be in the same track of thought.)

I voted "NO" to memory protection, as I'm judging from, "AOS works now, and great, I might add."
=============== post#71
Hi Everyone,


More concerns.


What is event tracking, it's different to memory protection, right?

Does it slow down the system much?

Can just event tracking be added without breaking any old SW?

Will memory protection keep demos from working, and democoders from doing anything that they could do or get away with in the past? Without knowing the other side of the coin, I'm kind of against it then.

Unless it JUST HAS TO CHANGE, I like the way it was/is just fine!!


BTW: Do embedded systems, PDAs, cell phones, STBs, etc. HAVE TO HAVE memory protection and/or resource tracking?? I could be wrong, but Hyperion don't seem to be concerned.
===============


So, what program CAN NOT be written if there is no memory protection available from the OS?

@All

Okey no fighting in this thread please and try to stay on topic. The topic is just fine to discuss, just don't go over board with angry replies or somesuch. Discuss it in a civil manner.

@Atheist

Why memory protection is important: imagine a program, which is buggy (all programs have bugs) and write into memory at random places. (Becuase it is using wrong pointers or whatever reason, there might be plenty.) If the system has no memory protection (like OS3.x) then it slips unnoticed, even the program might seem to work properly. But it has already placed the mines around the memory, for example it trashed one of the items in the free memory list. You quit the program and go on using your system, then once (when the system reaches THAT exact free memory node): BAAANG, the whole system goes down.

The memory protection won't prevent you doing everything which is legal on the OS. It prevents you doing illegal things. The program, which is not able to cooperate with memory protection MUST GO.
Demo coders must learn how to code system friendly finally. Can you point out any tricks which cannot be done with memory protection? (It's just lazyness, to tell you the truth, why l33t coders are doing such a crappy code.)

The memory protection itself is not slowing down significantly the OS, done by the MMU, the amount of data has to be moved around while task switching is minimal. (On PPC, other processors might have different needs.)

Speaking from experience: memory protection decreased the number of required reboots on OS4. This is a fact. The system is more stable, the badly behaving applications kicked out, even it helps the developers to track down hidden bugs.

@Atheist

If you ask "What MP is useful for?", I'll answer "To gain stability".
The fact is that developers are humans so they can make mistakes. So their software "can" crash. If they do, MP saves the system.
Of course, a program can be so well written that you can reach high degree of stability (even without MP). Do you remember that NASA used Amigas with programs they wrote?

But, in a market where you must release your software before your competitor, often you cannot work enough on your software. So the risk that it crashes is higher.
Having MP in the OS makes it more stable in the eyes of the end user.

I am sure that if M$ had not put MP in Windows, today there wouldn't be Windows anymore. It would have been replaced by Linux or MacOS.
But with MP, WinNT was already more stable than Win98. As a proof of that, M$ made NT Workstation. They knew they would sell it. And I worked for a company that used only this version on all client computers.
And then they generalised MP to the desktop version and made Win2000.

When my company (250 people) switched from 98 to 2000, I had a 1/3 of calls for assistance. I had enough time to use UAE all day long!

@rachy
Quote:
The only problem is, the application that crashes most these days here is IBrowse. When it crashes, I can't start another copy, because it detects that the old, crashed version is still there and tries to open another window on it. So I have to reboot anyway. Or I'm writing a program and it crashes, I can't compile a new version, because the old executable is still used and locked.

What is missing here is resource tracking. I guess a good implementation of this will slow down the OS much more than memory protection, wouldn't it?

So my idea: when or if resource tracking will be implemented, why not activate it for applications which the user can choose, like the blacklist for Petunia? I certainly don't need resource tracking for software which is running perfectly stable. Once I know that a program crashes sometimes or I'm currently writing a new program, I put it into the list, it will run a bit slower, but I can remove it completely and start it again without rebooting the OS.

Alex.

rachy wrote a great answer, well explained. Of course, a system can work without memory protection but that would be ridiculous to make AmigaOS without memory protection. The MMU is here hopefully and we can (we must) use it.

And from a developer point of view, it helps to track bugs. Without MP, as rachy told, a buggy program can cause a crash later. How could we find the real cause in a such situation ?

Resource tracking allows the system to know each resources are allocated by each program (memory, files, devices, ...) so if the program crashs, the system can kill it closing all resources properly.

@Atheist

as some ppl tend to think i'm not civilised, rude and all, i'll try to write this post with simple words, in civilised manner ;)

memory protection is a mechanism that strenghten the OS, giving it ability to kill buggy applications *before* they do something bad and crash down the whole OS.

it's a mechanism that monitor memory accesses and strongly forbid applications to endanger the OS by writing or trashing OS resources.
by closing or killing the programs unwilling to comply or behave as they should.

so to summarize: it's a process that monitor applications and if an application try to do something bad that could arm the OS, then the OS say "no you can't do that" and if the application insist it's being killed. and the system can continue to work as if nothing happened at all.

it means no more crashs. no more reboot. strong and stable system. and as rachy and other pointed out, if the memory protection layer is well done (the amiga way: fast, tiny and efficient) it won't slow the system down.

so applications will either need to be updated to support this new mechanism or will need to run in an isolated/quarantined memory space, to ensure they won't trash the OS.

it's no big deal. and the gain for user will be priceless.

I agree with elwood, memory protection is necessary to improve stability. When I had they XE, I had to reboot quite often, especially when Ibrowse crashed.
I think Windows is a good example for the benefits. With Windows 95 and 98 I had many crashes and blue-screens that took down the whole OS, so I had to reboot. This was annoying and of course it took longer than with OS4.
But with 2000 and XP I never had these crashes that locked the whole OS. If an application behaves badly (rarely happens) then you can just quit it with the task manager and restart it. It also doesn't affect your data which you have opened in the background, for example Microsoft Excel.
I am not sure if I would risk to write my thesis with OS4 at the moment. It is possible, but would have to backup and safe the documents very often, especially when I use IBrowse at the same time to google for information.

So if AmigaOS wants to be taken serious as an advanced operating system, then it needs memory protection. If it only wants to be an OS for remembering the past and to dwell in memories, then it of course doesn't need it. But then again, you could easily use your Amiga 500 instead.

P.S.: I dont understand what the big fuss on other webforums is about and why they are making a scandal out of it. I though it was clear for a long time that OS4 will get MP. The comparisons with MOS is only to make trollings.

In addition to the protection aspects mentioned by others (protection from buggy programs, protection of system structures, protection of data that is security sensitive), there's another thing: available address space

If you run each process in a separate address space, you have the full address space available. This means you can do nifty things like allocate a large portion of address space for the stack and automatically map physical pages to the stack as it grows.

Address space is also necessary for memory mapped files: Imagine a 1 GB file that you want to memory-map... in a single address space, chances are you won't be able to get 1 GB of address space in one piece, so it can't be mapped.

@ atheist

Quote:

All programs can be written with memory protection. I don't know who the original poster was, but he's totally off the mark.

What most people are "afraid" of when someone says "memory protection" is that they think that message passing can no longer be done with just passing a pointer around. Most people don't see an immediate solution to this, but it can be done (Mach does it).

Another thing people are afraid of is that they thing tightly integrated programs can no longer run multiple tasks and have them access the same data. This is of course completely invalid when you consider threads.

Regarding slowdown: On some CPU's an address space switch adds an overhead. This is true for example on the x86. OTOH, on the PPC, it's not the case since the PPC simulates a 56 bit address space (on the 32 bit CPU) so switching an address space is nothing more than reloading some registers (no need for any cache/TLB/MMU invalidation).

Quote:

Well, this is a rather nonsensical thing. Of course, any software that works correctly can work without protection, since the software doesn't violate anything. however, we all know that NO software is 100 % correct. Additionally, even if a program is 100 % correct, that might not be the case for other programs.

OTOH, and I take isolated address spaces as "memory protection" again, consider that some program might need more address space than you have... physical memory is not so much of a problem, just add a large swap partition, and the problem of physically available memory all but vanishes, but in a single address space system, you still have the problem that all programs are limited to their combined address space use (see the 1 GB memory mapped file example).

So bottom line, you CAN say that an isolated address space allows you to run more programs than a single address space.

Quote:

Not necessarily. OS4 can track resources already, but the problem is that a lot of resources are not allocated in the task that actually use them, so if the allocating tasks ends, it would kill off all resources that have been allocated, including those used externally.

Isolated address spaces also help in cleaning up: Non-shared resources will be mapped only in the particular address space, and when the address space is deleted (because the program quit or crashed), all resources are deleted, too (for resources shared among processes, this has to be done differently, of course).

It would also mean that if one thread inside the process crashes, it would kill the whole process (unless the process/thread can handle the crash, of course), making it possible to just restart it.

Quote:

You don't know which programs will actually support that... Ibrowse might or might not...

@ Helge
Quote:

Well, the fact that this was taken from here to another site only shows that the trolls are out again furthering their agenda. That's the usual thing, always happens...

What is the definition of "Memory Protection" ?

are you working on a concept similar to "protection bits" for files in a filesystem ? (allocated areas in the memory system...)

for me "common sense" is not common at all so I find discussions
about such things as this become convoluted and a mess

Id like to see a form of protection bits for memory areas,
as I would need such options for a "Symmetry" extension to
the exec.library

in addition to the above it would allow me to handle some
virtual machine management tasks in a simpler manner

Im still working on concept design only at this time,
until I can sort out some means of testing

anyway I'll leave the floor open to others before
I get completely sidetracked :)

Where's the original post that started all the nonsense at AW.net? The stuff I see here this morning points to the AW.net stuff, and I haven't found the origin of it all back here yet...

@Atheist

Quote:

In an ideal world perhaps. But this world is not ideal. There are malicious people out there. Even when they know exactly what they're doing, it can be messing up your system. memory protection can help in that context to some extent. It is also useful if one person is debugging software on a machine that a second person is logged into. If unfinished software being tested crashes, it should not interfere with whatever that second person is doing. Heck, it shouldn't interfere with other things the same person is doing. If his first Hello World program somehow crashes, it should not stop him from listening to mp3 music. Also, if a program crashes, it should not also crash the disk drivers and filesystem tasks and thus invalidate your drive or partition. That really sucks, and is one thing I'd like memory protection to prevent. My last go-through with an invalidated drive took 3 or 4 weeks to recover from. I don't want to be stuck with that much wasted time again due to some stupid crash!

@billt

Quote:

I think it was on this forum in the ACK thread, which was later renamed to something nicer. Rogue posted his ideas about memory protection there and then some well-known troll made a big fuss on Amigaworld about it. At least that's how I see it.

@GregS

Quote:

That's also how I understood it. But Petunia will still be needed to run 68k apps, or not? It would be a shame to drop it, much work has been put into it. Maybe the Petunia-JIT-emulator can be run inside the Virtual Environment together with the then obsolete OS4? I can't find a reason why not. So there is still emulation to some extent.

@Helge

I don?t think support for OS4.0 binaries will be need for where long, but that?s my option.

@LiveForIt

You might be right in that, i think it depends WHEN it will happen. If it happens very soon then there is not much problem. But if the cut happens very late, there will be a lot more OS4 binaries for which support is needed. Especially in the beginning, not many applications will be available for OS5 and you will have to resort on the OS4 stuffs.

@Atheist
As someone who has worked without memory protection (VxWorks) and with memory protection (Linux) in the real world on countless customized embedded (mostly PPC-based) boards I can say without reservation that memory protection makes your life easier.

After working many hours into the night on yet another memory trampling issue without memory protection on VxWorks I know the pain well. We introduced all sorts of things to make it less painful including but not limited to rigorous code inspections, various memory tracking tools, off-target running in a memory protected environment (Solaris), extensive automated unit tests and adding partial memory protection to VxWorks itself. Even with all that some errors slip through and you end up with a memory trampler that only shows during stress testing.

In the Linux world memory protection is enabled and memory tramplers have still occurred. Why? Because only processes are memory protected and not threads. Threads run within processes. So instead of a system wide memory trampler you end up with a sub-system memory trampler. Much less scope to worry about and much less energy spent to debug but still a pain.

Performance wise we were more than willing to pay a little extra for the memory protection for the massive savings due to shipping products on time. It is all about economics in the real world and it just makes sense.

I noticed some try to push no memory protection as a positive feature. Gotta love marketing. The reality is that we need to ship products and memory protection gets us there faster. Paying a little more for the quicker chips is a lot cheaper than working without the safety net that memory protection offers.

Partial memory protection is also *extremely* useful so don't think Amiga OS is totally out of the game. We used similar tricks in our VxWorks implementations and they saved countless hours of debugging. For example, I think the idea of MEMF_SHARED versus MEMF_PRIVATE is a good one and will only help move things forward and make it easier to build products with Amiga OS.

@ssolie
Quote:
Can you tell me what kind of allocation is made with the standard C(++) functions like "new", "malloc" or "calloc"?

Alex.

@alexw

Most likely MEMF_ANY or MEMF_SHARED

@Atheist

Memory protection actually helps coders write better code.There have been many cases of software working by luck, because they just happen to not overrwrite critical memory areas most of the time. With memory protection in the loop, the system tells you where you went wrong.

Added to this, you can't trust all coders. Without memory protection, malicious coders have so much more that they can do. With memory protection, the potential for deliberately messing up the system is less.

Hans

@Hans

?malicious coders? I don?t think there are any left where did they go to? Microsoft

probably working for bill gates now

@alexw

Quote:MEMF_PRIVATE