Max. partition size in AmigaOS4.1 (SFS2 handler)

This site tells me SFS2 has a maximum partition size of 1 TB, but that cannot be, since i'm using a 1.8 TB where 1.6 TB is available to the system.

Does anyone know how much is possible?
I' m looking to install a a new data grave and would like to make it as big as possible

Thanks a lot

@Raziel

Perhaps you only notice it when you retch above 1TB of stored files

I think the max size depends on your set block size to as well, did they take that into account when set max value?

@LiveForIt

I had that specific hdd 90% filled earlier, no problems, so I guess the max size *might* be 2 TB for SFS2 formatted devices, but there is no info I could look up.

What would the block size have to do with max partition size?

Since I cannot use NGFS due to it being held back for release for unknown reasons I'm stuck with SFS2 (which isn't a bad thing per se, just there doesn't seem to be much up-to-date information available nowadays)

I'd go with a 2 TB drive, but would like to know if larger size will/can work?

Thank you

@Raziel
http://os4depot.net/?function=showfil ... driver/filesystem/sfs.lha
Quote:That could be wrong and the actual limits are 2 TB (512 bytes/block) - 128 TB (32 KB/block) instead.
Maybe I've used 1-64 TB in the docs because I wasn't 100% sure there are no signed data types/calculations left in the SFS sources.
But I never had such a large HD, i.e. it's untested in either case.

Quote:File systems use block numbers to access the data, if a block is 512 bytes a 32 bit block number can access 2^32 blocks * 512 bytes = 2 TB, with 1024 bytes/block it's 2^32 * 1024 = 4 TB, ..., with 32768 bytes/block 128 TB.

Quote:Even if the 1 TB partition size limit is correct you could simply use 2 1 TB partitions, or 4 512 GB partitions, etc.
The HD size is limited by the AmigaOS device API to 16 EB (= 16384 PB = 16777216 TB ... = 2^64 bytes).

@joerg

Thank you, that help a lot

@joerg

Some more questions, if i may?

Does it make any difference speed-wise (access, i/o) if i'd adapt the block size depending on the maximum partition size used?

e.g. if i have a partition with 1 TB, would it make a difference using a block size of 512 (2 TB max size) instead of 256 (1 TB max size)

And do block sizes impact on ram usage to manage those partitions or is that only with the size of used buffers?

And finally, do buffers still impact on access speed with SSDs or just mechanical HDDs?

Thank you very much

@Raziel
Quote:If diskcache.library is used: No.
Even without diskcache.library SFS still has it's old, internal cache system which is used instead, but that's using smaller cache line sizes and much less memory for the caches and there may be a difference if 16 or 32 KB bytes/block are used for reads/writes from/to files, but OTOH using such large blocks will slow down other parts of SFS a lot, and with large block sizes small directories, files, etc. will use much more disk space.

Quote:There is only a difference for the buffers. You set the number of buffers in MediaToolBox. 1 buffer = 1 meta data block (directory blocks, bitmap blocks, trees, etc.). Buffers aren't used for the data blocks of files.
For example if you use 5000 buffers on a 512 bytes/block partition that's 2.44 MB RAM (5000*512), if you use 5000 buffers on a 4 KB/block partition it's 19.53 MB RAM (5000*4096).

Quote:Keeping often used blocks in RAM is always faster than having to reread them several times from a device again, even with the very fast nvme.device on a X5000 using memcpy() from a RAM buffer is still faster than (re)reading it from the device for the small transfers used for meta data blocks.

@joerg

Awesome, thank you so much for taking the time

@joerg

Ok l, so I installed the 6TB drive
...whatever I try, it's capped at 1.5TB.

I cannot change the "Logical size" in RDB/disk geometry...its set to 512 block size.

I have no idea if I could change anything in the expert section to make it work.

Funny enough, both Media Toolbox and Format report 5569GB, but as soon as the drive is formatted its getting capped.

No change in the sfs/partition setting helps either.

Any more advice/help, please?

Tia

@Raziel
Quote:I don't know anything about the Sam460, X1000, X5000 and A1222 drivers, but the old ones from sg2 (a1ide, peg2ide, Sam440 and the PCI PATA, SATA and SCSI ones) don't support anything other than 512 bytes/sector for HDs.

Quote:How do you check the size? C:Info, Workbench, etc. may not support displaying the size correctly for partitions > 4 TB as they would have to do some workarounds themselves, FileSystems can't return correct sizes > 4 TB for 512 bytes/block partitions because in functions like IDOS->GetDiskInfo() id_NumBlocks is 32 bit.

Try if using two 3 TB partitions display the correct size.

Another option might be to use a single 6 TB SFS\2 partition with 1024 bytes/block, but using 2 3 TB partitions with 512 bytes/block is better.

@Raziel

There is disk physical block size which you cannot change in the drive geometry window. Because this a property of the disk itself. And Media Toolbox only support 512 bytes anyways.

But you can change the filesystem blocksize when you create a partition.

@geennaam

I know about the partition changes, but they won't help.

So if media toolbox doesn't support anything else than 512 bytes we are stuck at 2 TB partitions anyways?

@joerg

All of the AmigaOS4 tools seem to show the correct size (5589GB is the correct one, btw) but as soon as the disk was formatted its being capped.
It looks like the problem is still with the fs(?)

And I did exactly that, a full hdd partition (5589GB) with 512, 1024 and 2048 blocks, none will go beyond 1.5 TB

I'll try the two 3TB partition tomorrow, but I don't think it make a difference.

BTW the info part of mediatoolbox shows a -999999999 at sectors I think it was...calculation error, probably a buffer overrun

@Raziel

Those two blocks sizes are unrelated.

Trackdisk64 standard allows you to access a 64bit address range.

An SSD/HDD is not address with bytes but with logic blocks. This logic block size is traditionally 512 bytes. So instead of writing an amount of bytes starting at a certain address, you write an amount of logic blocks starting at a certain block number. This means that the file system needs to address the SSD/HDD and transfer data in multiples of logic blocks. This is called logical block addressing (LBA). A Nvme drive can address 64bit of those LBAs meaning a theoretical maximum storage size of 64+9=73bit in case of a 512 block size.

Now the file system itself can use a different block size as data container to organize the storage size of a partition. The only restriction here is that the file system block size must be a multiple of the SSD/HDD LBA size.
Larger containers are more efficient from a transfer speed point of view. But will also lead to more waste of storage because you get more containers that are not fully occupied with valid data.

Trackdisk64 might be able to address a 64bit address space. Bit apparently the SFS2 partition size is addressed with 32bit. 32bit with a file system block size of 512 bytes results in a maximum partition size of 41bit (=2TB). A block size of 1024 bytes results in a limit of 4TB and so on.

Modern sata drives and Nvme drives might also support logical blocks of different sizes. For example 4096 bytes. I actually have a Nvme drive which allows me select 4096 bytes LBA. But unfortunately Media Toolbox doesn't allow me to create partitions in that case. Because it does not support any other LBA size then 512 bytes.


The size that Media Toolbox shows you is reported by the driver. That's simply amount of LBAs x LBA size.
The partition size limit could be the result of another unknown limit inside Media Toolbox.

You could try FFS2 instead. Just to be sure.

@geennaam

https://postimg.cc/gallery/NtV8TJM

FFS2 doesn't go as far as SFS...i can format the 5TB partition, but Format errors out with a "Disk full".

See also the correct calculation on all programs (there is a red dot in the RDB mask, though, can't change it to green)

I have yet to try the two 3 TB partitions, but i believe it will still cap at 1.5 (or 2 TB) nonetheless

...

So, you are saying that the used filesystem could use such huge disks?
Or is this OS level?

I wonder if NGFS is capable of using more than 2 TB (in real life, not theoretically)

@joerg

Ok, so that is interesting.

Splitting the disk into two 3 TB partitions (SFS/2) both partitions cap at 746,5 GB (half of the 1.5 TB if i use the whole disk).

It seems as if Media Toolbox(?) or SFS(?) is doing the housekeeping for the whole disk instead of the seperate partitions(?) capping everything from the same disk at 2 TB(?)

nb:
i have one other big hdd at 1.8 TB and this disk is completely used (probably because it's still flying below the 2 TB barrier)

Any more ideas?
Or can i scrap the idea of using the whole disk (anytime close/ever) under AmigaOS4?

@geenaam
@joerg

Partitioning the disk to two 2048 GB and one with the rest works perfectly fine (two 2 TB and one 1.5 TB partition)

So, i guess that's it?
We are stuck at 2 TB max hdds...too bad :-/

@Raziel

There's no conclusive evidence yet which part of the system is to blame. It can be the filesystem, Media Toolbox or even the driver.

Unfortunately I do not have such a large drive available for testing (yet).

Nevertheless, the Media Toolbox should be fixed to support > 512 LBA size.

Edit: The "disk full"error after formatting happens when there has been write errors to the file system administration on disk. Happened to me as well while developing the NVMe driver

@geennaam

Ok, thank you for the info

I don't think i will live long enough to see anything getting released regarding Hyperion based AmigaOS4 stuff outside the beta cloud (apart from *years* apart updates)

@Raziel
Quote:Quote:Double the "Blocks per cyl." and halve the "Last usable cyl." values at the same time, repeat until both get green.

Please note: If you change anything there you'll loose all of your partitions and have to recreate them.