4GB RAM in AmigaOS?

AmigaOS is limited to 2GB of RAM, but why?

How much software would really fall apart when given negative (>2GB) memory addresses?

I mean who on earth in their right mind would do this:

ptr=AllocMem(...);
if(ptr<=NULL) {
printf("Error, out of memory!\n");
return false;
}

Or this:

win=OpenWindow(...);
if(win<=0) {
printf("Error, could not open window!\n");
return false;
}

Does it really exist any program (that runs on Petunia today) that does this horribly incorrect thing??!?

*** 4GB to the community! ***

I ordered 4gb for x1000.

Quote:

Then I guess there are plans for it at least

As the PA6T can address 32GB RAM and X1000 can hold at least 16GB, I imagine 64bit memory addressing (+the use of two memory bus) is on the roadmap...

For me it would be ok if they need to limit apps to use maximum of 2GB per application. But the rest should be available with future applications and for OS needs.

@deniil
I think the problem with 4GB pointers is not <= NULL, but rather pointer arithmetic. If someone uses signed rather than unsigned maths, then pointers above 2GB will go screwey.

e.g. 3000MB + 1MB would result in 2999MB (rather than the expected 3001MB), because 3000MB would be interpreted as -1000MB by signed maths.

Also APIs often return pointers using signed values, although I think Hyperion (and maybe even before them) have gradually been trying to get rid of these oversights with every SDK update.

Also there may be other issues such as Commodore defining certain address ranges (esp above 2GB) for certain uses. Might cause problems for some drivers?

Negative pointer values are some times used as error codes.

@LiveForIt:
Are you sure about that? I can't think of any examples where a pointer with sign bit set means anything special.

In many DOS calls, pointers have to be converted into BPTRs by ( >> 2). If the original pointer is > 2GB, it has a sign bit which is maintained during the shift. Now there may be some functions in DOS which don't check that the upper bits are zero and which would fail for that reason. I'm only guessing.

On the other hand there are many structures defined like this:
Quote:

Now fh_Pos and fh_End are set to -1 when undefined (when the file is not open). To use the whole 4GB address range, we would have to replace those int32s with a uint32 and a separate "invalid" indicator. Otherwise we would have to limit the available address space to (4 GB - a few values).

Quote:

Irrelevant. -1000 + 1 is -999 = 3001. Singed and unsigned math doesn't matter. It is only the final binary unsigned result that is important and that will always be the same regardelss of signess on a higher level. That's the beauty of 2's-complement binary.

Note that in assembly there are no signed or unsigned add/sub instructions. Only the branch instructions care about sign by looking at different flags (overflow/negative/carry) depending if they should interpret the MSB as a sign bit or as a value bit. That's why you shouldn't do "if(ptr<=0)".

I can see one place where it is a problem and that is shifting arithmetically with sign-extension. Like if you for some odd reason would want to halv or quarter an address (create an obsolete BCPL pointer for example using LONG instead of ULONG).

Quote:

A pointer can never be signed. Although if it is returned in a LONG (not ULONG which is what seems common where ptrs isn't returned as actual pointers) it would still be fine passing around. I just doesn't matter.

I find it very unlikely that some API returning a pointer in a signed long (or int(!) *shudders*) also expect the user to do arithmetic other than add/sub on it.

Quote:

What Commodore defined for drivers must surely be pretty obsolete for OS4/PPC. Even if some classic driver would have problem I doubt anyone would even be able to put more than 2GB in a classic in the first place.


@LiveforIt

Quote:

Where?
But those are then specific error values, like -1, -2, -3 (as constants), not negative in general.

@tonyw

Quote:

This is only true if the BPTR value is signed to begin with, i.e. a LONG instead of an ULONG. The compiler would choose the correct shift instruction based on the high-level type it is going to shift - to sign-extend or not.

@Deniil Quote:
Hmmm, yes, you are right. An example, using signed bytes for simplicity:
-16 + 1 = -15
%11110000 + %00000001 = %11110001
with unsigned bytes would appear as:
240 + 1 = 241

The (un)signedness merely determines how the bitpattern is interpreted. BUT that still means the following 'pseudo' code would break if it crossed the 2GB/-2GB boundary:

Quote:
e.g. If ptr1 is 2GB - 10 bytes, and ptr2 is 2GB + 10 bytes (i.e. -2GB + 10 bytes when interpreted as signed), then ptr1 would never be incremented, because ptr1 would already appear as below ptr2 on the first test.

Is there any typical C code where this would happen? (I prefer to avoid using C, so off the top of my head I'm not sure.)

@ChrisH

Quote:

A pointer is inheritly unsigned, unless you explicitly put it in a (signed) long before doing the arithmetics so this while loop works perfectly fine over the 2GB boundary. (I just tried.)

Interpreting the MSBit in an address as a sign bit is irrelevant since having negative memory is pointless. Sure you can design a machine to have the upper 2GB reserved for drivers or IO or DMA or kernel or whatever and the lower 2GB for user memory. But hardcoding a machine like that isn't very sane, and the Amiga used to have a 24-bit memory space to begin with so upper/lower 32-bit was irrelevant for other reasons as well back then.

As far as I understand the southbridge handle address ranging and how the upper bits should be interpreted and distributed over the motherboard. Of course the MMU has his to say before the address even leaves the CPU chip.

If the PA6T can handle 64GB RAM then it has a 36-bit logical address bus, like the intel chips with PAE.

Quote:
I know a normal C pointer is unsigned, but my question was supposed to be: Are there are cases in C where you are likely to put pointers into a (signed) long, etc? I guess not from your answer...

Quote:

By cooincidence I just read abit of code on the net that did


if(ptr > 0) { }

!!!!

So whilst it is a Bad Thing TM, some people do it....

@TonyW

I was guessing, I think I read some where that dos.library did this, maybe its dos packets I don't know, there has to be a reason way only 2GB can be addressed, the easiest way to find It out is try is it not, crank up virtual memory limit and see what breaks.

We can already address > 2GB. The only limitation at the moment is that the kernel deliberately won't generate addresses > 2GB (because we don't know what will happen). That restriction has to be removed at some point so that we can find all the places where things go wrong. There are bound to be some.

I don't know how big a change it would be to enable 4GB throughout the kernel. But is it worth the effort? It might be not much more difficult to rule the line under the current architecture and change everything to 64-bit in a new fork.

@tonyw

Quote:

Going from 31 to 32 bits is not quite the same as going to 64 bits..
64-bit will break everything. Every single struct that contains a pointer will become incompatible, and there are a lot of those! I'd say SMP is easier than 64-bit.

But testing for 4GB would be easy: Just have the kernel put ALL addresses in the upper region - memory, I/O, everything Make "positive" (lower) 2GB 'illegal' instead of the upper "negative" 2GB as it is now. It should be a setting so you beta testers can switch and test different software and drivers and stuff.

Should give you an indication pretty quick if something isn't up to it.

@ kicko

same here i've been burned too many times trying to match ram modules on old pc's so i figure it's always better to get a matching pair whenever you can

Deniil wrote:

Quote:

I agree - it would be nice to have full 4GB available. Assuming the OS itself is 32-bit clean any apps that are broken could be put into a compatibility database until newer versions come out.

As for 64-bit/SMP support, it would be nice if there was a "big bang" "V2" interface upgrade where the remaining warts (memory protection, forbit/permit, etc...) were taken care of at once. This is likely not possible due to the scale of the change but it would be nice to get over this in one big hump.

@ChrisH - only badly coded drivers would use signed pointer math on the 680x0 systems,
PPC based OS versions I don't yet know enough details to say anything...

@everyone...(@TonyW - exec.library AutoDoc AllocMem variation for multiple Allocations in a List)

see the exec.library AutoDocument for "AllocMem()" variants...

There is a **SINGLE** Memory Allocation libcall in exec.library blocking use of addresses above 2GB,
and this specific exec.library/_LVOAllocMemPooled()??? call documents the 2GB boundary as exclusively introduced
by this library call...

All Classic Kickstart drivers *DO*NOT*CARE* as long as the CPU can validly read and write the block of memory
within the standard 680x0 4GB of Addressable memory.

Any *3rd*Party* drivers such as diskimage.device, fmsdisk.device, gvpscsi.device oktagon.device,
as compiled by Lattice C or SAS C&C++ or the GCC compilers will not currently be using any kind of signed pointer math
produced from the compilers without an explicit declaration of signed pointer testing directly in the sources.

so ChrisH's comment about drivers would more apply to any additional drivers not provided as part of AOS2.x/AOS3.x releases.

There is ****NOTHING**** else stopping full usage of 4GB in the software at this time I am aware of...
THIS IS FULLY DOCUMENTED IN THE THIRD EDITION ROM KERNEL REFERENCE MANUALS.

However you do have to realize... the original Classic
AmigaOS is written on a processor where IO devices use
64KB blocks of memory as a minimum AutoConfigure block size

Additionally the memory map has the following...

2MB ChipRAM
8MB FastRAM (ZorroII?)
5MB Motherboard & Zorro2 IO space (ECS and AGA / CIA / IDE or SCSI functions)
1MB ROM Space (512KB Cartridge, 512KB Kickstart)

AutoConfig Occurs at a fixed Address below the 2GB watermark,

if you physically map 4GB of memory... without "holes" for the IO device spaces on the 680x0 series processors,
I understand from the 68000/68010/68020/68030/68040 and 68060 Manuals that this will trigger
"double bus fault" conditions... only ONE type of device can respond at any given memory location.

Double-Bus faults are a *lockup* condition so any type of device IO would halt the processor with only one interrupt
allowing a response (Interrupt 7, all interrupt lines set), this is the NMI or Non-Maskable.

These are the only two things blocking usage of memory...
if anyone wants to install 3x ZorRAM 512MB cards into any A3000 or A4000 Amiga they can quite happily use the entire
1.5GB of FastRAM on all three cards in addition to any other expansions present...

The total sum of memory installable on AmigaOS 3.9 and older is ~1.95GB (allowing for the IO device space holes),
however... THIS INCLUDES THE MEMORY FOR A GRAPHICS CARD AS WELL...

I have no experience or current knowledge of the PPC systems available... but I will definitely be exploring options
once I get my hands on a sam440flex followed by a sam460ex...

if anyone wants any further clarification on any of this information they can lookup the exec AutoDoc for
the Alloc*() routines and additionally the Memory Map as described in the various Hardware reference books...

AutoConfig itself *does*not*limit* memory to 2GB... it will actually quite happilly map a 2GB Memory board *STARTING*
at the 2GB boundary for use as Memory (but it may not be pool allocated)...

The AmigaOS exec itself is not limited to generating addresses above 2GB without a device being AutoConfigured above that
soft boundary...

But you had better be prepared for the condition of a false failure within the Allocator when it returns memory that IS
bit 31 set tested (this is the Soft Error condition for a multiple Allocation with a single libcall to exec.library)

To my knowledge there is only the hardware limitation for certain memory regions below 2GB that are mapped to IO not memory
so can anyone force a fixed Memory device to be added to exec through Adding the Memory concerned directly to the
Memory lists for testing purposes?

I'm more than happy to try hacking this into any of the big-box Amiga systems where this is possible...

Hell... anyone got a Zorro card which would allow wire-wrapping for experimentation? I'm game to try it...
only Classic systems with Zorro3 capabilities would be able to support such an experiment to my knowledge.

There is also a documentation reference with regards the 68000 TAS(Test-And-Set) Read-Modify-Write cycling of access to memory
being actively broken by the Amiga chipset (the chipset itself would need modification to support SMP for this approach)

so any kind of SMP option with 680x0 Classic Amigas would require use of the 68020+ "Co Processor Protocol" as documented
in the Motorola-SPS / Freescale 680x0 Documentation (more fully explained in the 68040 and 68060 User Manuals).

This would allow an FPGA that is setup as a custom Co-Processor (same protocol as the FPU and MMU chips but a different ID)
to actively respond or modify bus arbitration between processors... including HALT and Interrupt handling.

not a small project in itself but does require that the SMP co-processor isolates the CPUs from all other bus master capable
devices for both BIU and Interrupt Handling logic, possibly having its own limited register set for such things.

The Motorola Co-Processor protocol is 16bits per transfer and can deal with a 16bits per Word "discussion" with the limits
of 2 words for the instruction, and 128 words for immedate data operations.

I would need to read the PPC documents for the appropriate chips to find an equal solution there.

I'm sorry, belxjander, but most of what you said in that post does not apply today and has been obsolescent for the last ten years. We are talking about OS4.

The OS4 kernel restricts memory allocations to the 2GB limit, for the reasons already stated. I would be willing to bet that the system would still boot normally if we suddenly moved everything above the 2GB barrier (ignoring the fact that we still need access to the zero page). If we allowed memory allocations to cross the 2GB barrier, some stuff would work and some would not.

As to the task of changing from 32-bit to 64, it's not dissimilar to the task of changing from signed 32-pit pointers to unsigned. Someone still has to go through every line of code in every source module in every OS component, checking for side effects. If you are going to edit int32s to uint32s, it's easier to edit them all to APTRs. The real work is going through the code, finding all the places where something special has to be done. Once the naive assumptions in the code have been fixed, you can start the fun job of debugging it all over again.

The fact remains that we have better things to do at the moment.