Memory protection and tasks/processes

With the new MEMF_PRIVATE flag for memory allocations, I was wondering how this related to tasks/processes. On other OSes, processes have separate memory spaces, but threads within a process can share data.

So, if I declare memory MEMF_PRIVATE, can I access them from all tasks created by the same DOS process, or is it private to that task? It would be nice to have threads with the same memory protection model as other systems (i.e., protect the process, but allow threads to share data freely).

I'm about to start work on multi-threading, and it would like to use what memory protection facilities are available.

Hans

@Hans

AFAIK, at the moment you'd better use MEMF_SHARED memory between your different processes because parent process memory flagged as MEMF_PRIVATE is not shared with its children (NP_Child flag) (someone correct me if I'm wrong).

@centaurz

Quote:

When you say process, do you mean a DOS process, or a task? We're going to have to be very specific with our terminology, or things are going to get very confusing. I'm asking about multiple tasks created by one DOS process.

Hans

@Hans

I mean DOS process, the parent/child feature is available through DOS function CreateNewProcTags().

@centaurz

It looks like memory protection works at the task level, not the process level (i.e., it has no concept of DOS processes). IIRC, the current pthreads library is a wrapper that uses tasks/processes, so that wouldn't help either. As long as there's no concept of child-tasks or threads, I don't think that threads sharing local memory that is protected from other processes is possible. Hopefully this will be added at some point.

Hans

@Hans

AmigaDOS processes have always been an extended for Exec Task,

@all

I personally see it this way... any "Exec Task" no matter who launched it
has its own memory space, and only "MEMF_SHARED" memory is going
to be available between them.

tasks/processes/threads/ does it really matter...
other OS may allow threads to occupy the same memory space,
but we are not those other OS, why not take the path of
every seperate thread of execution as being an own task whether
able to be an AmigaDOS extended process or not?

that means if it is shared amongst threads/tasks/process then mark it shared,
otherwise it isnt available, seems to be the simplest rule to follow

if anyone more knowing about the subject can correct me for AOS4 specifics?

@Hans

On AmigaOS the only difference between a "task" and a "process" (from Exec's point of view they are all tasks) is that the "struct Process" contains some additional fields over the "struct Task". Most notably the pr_MsgPort which is used by dos.library for communicating with filesystems.

The only real difference between a task and a process is that a task can't make use of DOS while a process can.

@Belxjander

Quote:

Forget the separate memory spaces. That's not the issue. The reason for wanting to share memory between threads within the same process is to simplify inter-thread communication. I'd like to prevent other applications from screwing with memory belonging to my app, but still allow free (but mutex protected) sharing of data between threads within my application. This doesn't require separate memory spaces as Amiga OS is a single memory space OS. It would require a concept of child-tasks (and possibly a new memory protection classification).

Hans

@thread

Is MP anywhere to be seen in the AOS4 "roadmap"?

Pre study, kind of:

"Will AROS have memory protection, SVM, RT, ...?
Several hundred Amiga experts (that's what they thought of themselves at least) tried for three years to find a way to implement memory protection (MP) for AmigaOS. They failed. You should take it as a fact that the normal AmigaOS will never have MP like Unix or Windows NT.

But all is not lost. There are plans to integrate a variant of MP into AROS which will allows protection of at least new programs which know about it. Some efforts in this area look really promising. Also, is it really a problem if your machine crashes? Let me explain, before you nail me to a tree. The problem is not that the machine crashes, but rather:

You have no good idea why it crashed. Basically, you end up having to poke with a 100ft pole into a swamp with a thick fog.
You lose your work. Rebooting the machine is really no issue.

What we could try to construct is a system which will at least alert if something dubious is happening and which can tell you in great detail what was happening when the machine crashed and which will allow you to save your work and then crash. There will also be a means to check what has been saved so you can be sure that you don't continue with corrupted data.

The same thing goes for SVM (swappable virtual memory), RT (resource tracking) and SMP (symmetric multiprocessing). We are currently planning how to implement them, making sure that adding these features will be painless. However, they do not have the highest priority right now. Very basic RT has been added, though." - by AROS devs


Weird version of MP: (a few thoughts while having a cup of tea)
1) MP spaces for new apps (old apps could still kill the OS)
2) possibility to rewoke MP apps after AOS+old apps crash (only legacy apps + data would be lost in crash)
3) moving AOS services gradually to MP area
4) ...???

Added with other SW shutdown + restore features, these could improve the useability of AOS @ work.
http://amigaworld.net/modules/newbb/v ... 160&viewmode=flat&order=0

@KimmoK

Maybe you missed it, but this concept is reality since the GrimReaper was invented.

The problem with crashes is not the fact that a single Process/Task crashes, its the system itself. Even there some cases may occur where MP wont help a single bit. Theres a reason why BSODs occur, or kernel panics.

One could do a big effort to ensure that crashed kernel space processes are killed and restarted, but I strongly believe that this is not a way AmigaOS should go. Its the way to "bloat".

IMHO there should be much more effort in IDE things. We miss a useable debugger. We lack lots of good information about the OS internals (no, internals should not be hidden from application developers. The devs should be encouraged much more not to abuse them by examples how to not abuse them!). Good help is given by sites like utilitybase, or this site here.

This help should be expanded much more, massive information interchange is the key here. It will lead to much higher software quality than MP could do ever. Have a look into other platforms, their driver bugs and quirks, and you will know what I mean.

@whose

Valid points !

So we have more important challenges before "MP" -> full MP most likely is not in the top 100 of the feature list/roadmap.

@KimmoK

There is no replacement for quality software. Adding features to the OS in order to stop badly written programs bringing down the system will only encourage more badly written programs.

The Amiga has always been about careful programming and optimising, and this is what made it lean and mean years ago. There is no reason why this should change.

@thread

Opening up the internals to the application programmer also encourages the use of features which are not guaranteed to hang around. A typical example here is the use of the memory block stored in the LONG preceeding the actual block allocated. While this was never documented, coders used it, and now we have situations where users complain because that software no longer works. Allbeit, lots of workarounds have been included to allow some of these rogue programs to continue to function, but there has be a line drawn on how far to go, as most of these workarounds stop progress of further (more important) features being implemented.

As most 68k software was written long ago, most has been since been surpassed by modern software to take it's place. Admittedly there are some applications that are still used daily, but there should come a time where 68k compatibilty will be "switched off" IMO, this then allows more progress rather than hanging onto 15 year old applications.

The more vocal "supporters" moan because AmigaOS is still based on "17 year old APIs", but until the workarounds in them can be finally done away with, what can be done? If they are removed too quickly, users will moan that the "modern" AmigaOS has no compatibility. Unfortunately, the lack of developers holds up this transistion, we need quality software to replace the old classic applications of yore, perhaps then we can start to move forward a bit quicker. AROS has the obvious advantage here, as it doesn't have to care about these old programs, a move which OS4 could not unfortunately afford to take. Shame.

Simon

@Rigo

Imho better to remove 68k from OS itself at all, and just make E-UAE better (integrate Petunia parts to it, to make JIT for E-UAE), and integrate it in the OS like glUAE or RunInUAE, or by any other and good way. Then, it will be just the same as it now, but os4-core developers will no more think about 68k at all. I see none real advantage to have 68k support, old API support, etc. It's all stop progress of aos4.

Personally i am, running none of 68k programms on my os4 today. Just few demos/games , which i run over EUAE already. For 68k just need fast working and good integrated by default E-UAE, and imho no need to have the same, but inside the OS.

@Rigo

Quote:

Sorry to disagree here Rigo, but that's just nonsense. If your OS stops you from doing bad stuff outside your memory range etc., the developer will see it immediately during debugging/development. If the OS does not have such feature, the developer won't always notice and the bug appears only for other users, making it hard to track down.
The OS should do everything and I mean everything to prevent taking down the system.

@DaveAE

I'm not saying don't implement it, I'm saying don't rely on it.

Properly written software will do more good than preventative measures. It's hardly like AmigaOS is "Mission Critical", is it?

Simon

@Rigo

Mission critical or not, you can only write good software if you got the tools to do it, and one of those is an OS that helps you point out nasty things.
Even expert software developers make bugs hourly/daily. How do you know if you write good software if nothing is saying that you are? For example, before OS4, you could happily write to address 0 and not be notified and you would think you wrote good software. Now with OS4 you are immediately reminded that you didn't write good software at all.

@Rigo

Imho, relying of properly written software is far worse that relying on the OS to keeps things in check.

Just to prove how bad it works in reality: there has basically never been any software at all for AOS that has been properly written.

And here's a few reasons why (there's more but I don't have the time):

- There's no vetting procedure to make it happen
- There's no one with resources available to vet their software to the point that it becomes properly written
- The OS does not offer enough support for developers to be able to write properly written software
- There isn't and has never been enough documentation available to write AOS compliant "properly written software"

@orgin Quote:
I basically agree with Rigo, although I don't think I'm too far from your position either:

AmigaOS can be *incredibly* stable, but only if you avoid buggy programs. I found OS3 very stable (until I went PPC...) by avoiding buggy programs. So it can work in practice, as well as in principle.

That's not to say better protection at the OS level is a bad thing. It's good if you can add it. But as long discussions here will attest, it isn't easy to add such things to AmigaOS, so we are left avoiding buggy programs as the final solution.

If OWB had one major bug fixed, and SimpleMail had one bug fixed, and I didn't develop programs (which involves lots of bugs!), then I'd probably find OS4 to be very stable. As it is, it's stability is just "good enough".

@ChrisH

But, when the rest of the world is using incremental compilation to optimise their coding, we rely on make, run, reboot.

Obviously not every time, but quite often, and gdb didn't help me avoid that cycle last time I used AmigaOS for development.

@orgin

Well, but atm. we have the tools to check the software. Are they really used? How many programs are out there throwing a GR which is not fixed?

Another one: Any of you experienced these "interesting" quirks with some software on systems with full MP? I do each day and I see, that full MP leads most coders to do it with a "well, at least it doesnt crash" attitude.

Not really the path to HQ software, dont you think?

Oh damn, we hijacked Hans? thread