Help with HDF Files

Hi,

Can anyone give me some help on hdf files.

1) How many can E-UAE support in one configuration?

Documentation only lists 'hardfile' as an config option (and 'hardfile2' for specifying the geometry) whereas floppy0, floppy1, floppy2 and floppy3 are for 4 different disks (etc). Can I have more than 1 hardfile?

2a) What is the largest size file it can support?

Nothing in docs. Conflicting reports on the web of 1GB or 2GB. Have tried both, but I think that using the default geometry (32 sectors, 1 head, 2 reserved blocks) translates into too many tracks or something... don't know but it doesn't work.

2b) What is the geometry for this size disk?

Is there a geometry calculator out there somewhere I can use to work this out for myself?

3) How can I make a disk file which appears in the HDToolBox?

Do I have to use RDB disk images? I couldn't find anything particularly useful with creation of such.

I'm running E-UAE 0.8.29-WIP4 on Linux.

tiffers.

@tiffers

Quote:
Up to 30 hard files and/or virtual filesystems combined are supported.

Quote:
On systems which only support 32-bit file sizes, the maximum hardfile size is 2GB. On Linux, there is no practical limit, however. I suspect the limiting factor would be the maximum number of cylinders that AmigaOS supports in a disk. I'll need to check that.

Quote:
The size of a disk is defined as:

size = block_size * sec_per_track * surfaces * cylinders

Block size is typically 512 and common practice in UAE is to set surfaces to 1 and sectors per track to 32. Then for a given hard file, UAE will work out the number of cylinders based on the size of the hardfile:

cylinders = size / (512 * 32 * 1)

The next version of E-UAE will include a makedisk tool to simplify the process of creating hardfiles.

Quote:
Yes. RDB hardfiles are partitionable with HDToolBox.

@evilrich

Quote:

So, in theory, doubling the number of surfaces will half the numbers of cylinders. Is that correct? In fact doubling any one of those parameters (bytes per sector, sectors per track, surfaces) would half the number of cylinders.

Obviously 512 bytes per sector is pretty standard. I would think that increasing heads (number of surfaces) and sectors per track would be the trick. This is kind of what I tried, but every combination I tried didn't work. Needless to say I didn't try every possible combination ;)

Maybe some more trial and error will help.

How hard is it to define an RDB disk? How do I ad extra hardfiles? Just add more lines starting with 'hardfile=' ??

tiffers?

@evilrich

Quote:Sectorsize, SectorsPerBlock, Surfaces/Heads, SectorsPerTrack and Cylinders/LowCyl/HighCyl are all 32 bit values, the file systems don't use them in normal use but only when they are started and convert them to 64 bit byte start- and end-offsets and usually a 32 bit number of blocks for the size of the partition. The limits are:
- The block size (SectorSize * SectorsPerBlock) has to be 512, 1024, 2048, 4096, 8192, 16384 or 32768, some file systems support all multiples of 512 up to 32 KB, some additional sizes, for example very old FFS versions even supported 256 bytes/block.
- The max. partition size is between 2 TB (512 bytes/block) and 128 TB (32 KB/block), nearly all AmigaOS file systems use 32 bit block numbers. Some file systems may have smaller limits, for example in SFS the max. partition size is only 128 GB, no matter which block size is used.
- With the AmigaOS 3.5+ FFS and file systems like SFS the max. HD size supported is 16 EB (64 bit byte offset in the device API using the newstyle commands).
- With the FFS included in AmigaOS <= 3.1 and other old file systems the max. HD size supported is 4 GB (32 bit byte offset in the device API using the old commands). Most old file systems don't have checks for this limit, if you use them on a larger HD (or imagefile in UAE) and for example try to access something at offet 5 GB of the HD they access the HD at offset 1 GB instead, i.e. when writing they destory data on other partitions.

@all

I have tested some of the ideas I discussed above and can confirm:

1) with the geomeetry of 32,1,2 the max disk size was 2047MB (2048MB didn't show)
2) With geometry of 32,2,2 and doubling the disk size to 4094 MB worked.
3) A 8188 MB image with geometry of 32,4,2 was only recognized as 4092 MB, so 4GB seems to be the limit.

I was using OS 3.1.


It seems my previous failure to make disks bigger then a few hundred MB was due to:

1) Running UAE (not E-UAE) that I had compiled from sources
2) That UAE binary didn't have any SCSI capaility enabled, which is what is used when you specify a geometry (apparently)

I'm still keen to know how to make an RDB image.

tiffers

@joerg

Quote:
Thanks for the information. That saved me some time!

Cheers,
Rich

@tiffers

Quote:
Create an empty file of the desired size. For example:

dd if=/dev/zero of=~/hardfile.hdf bs=512 count=163840

would create an 80MB hardfile called 'hardfile.hdf'.


Then add the option:

hardfile2=rw,:~/hardfile.hdf,0,0,0,0,0,

to your E-UAE config file.

Note that you don't specify a device name or geometry (this info gets stored in the RDB when you run hdtoolbox).

Before you can use an RDB hardfile in AmigaOS, you need to initialize it then partition it with HDtoolbox or similar - just like you would a real disk. Get HDtoolbox to scan uaehf.device.

Just checked this right now, and I did find one problem. I had a filesystem-style hardfile mounted as well an RDB one, and HDtoolbox hung on the former. If you run into this problem, you may need to temporarily disable any filesystem-style hardfiles until you've partitioned your RDB one.

Cheers,
Rich

Quote:
Actually, the problem is more serious than that. There was a race condition in the hardfile code. I've fixed this now. See
this mail for a patch. The fix is also in CVS.

For those who don't want to build from source, I plan a new binary release in a few days containing this and other bug fixes.

Cheers,
Rich

@tiffers

Quote:2047*1024*1024/512/32/1/2 = 65504 cylinders
2048*1024*1024/512/32/1/2 = 65536 cylinders
4094*1024*1024/512/32/2/2 = 65504 cylinders

I checked the old sources and release notes of FFS and the AmigaOS <= 3.1 versions had several bugs, for example it's using mulu.w (HighCyl-LowCyl+1), SectorsPerCylinder to calculate the number of sectors on a partition, but since mulu.w is a 16 * 16 = 32 bit multiplication that can't work ...

In the AmigaOS 3.5 FFS release notes (version 43.11) there is
Quote:but that doesn't help much since the majority of the bugs was only fixed in the AmigaOS 3.9 FFS, version 45.4 release notes:
Quote:
Except for floppy images such single partition image files don't make any sense to me, especially if they always use FFS and you can't select the DOSType of the file system. Unlike a RDB image file you could "eject" them and "insert" another one while UAE is running, but that only makes sense for floppy images as well. Or for emulating other removable media drives like MO or ZIP.
If you can't select the DOSType for such image files IMHO UAE should check and reject them if they use a geometry which exceeds any of these 16 bit limits of the old AmigaOS FFS versions. Using AmigaOS 3.9 doesn't help since it has to be loaded using the AmigaOS 3.1 kickstart ROM version of FFS. For some image files UAE could change the geometry to work around these 16 bit bugs, but that's not always possible and simply rejecting them with an error message is easier.

Quote:8188 % 4096 = 4092, looks like a 32 bit overflow. With AmigaOS 3.1, and without using a 3rd party file system, you are limited to 4 GB anyway, but that may be a limit in the UAE binary you are using if it was built using 32 bit I/O functions, or if it's built with 64 bit I/O functions a bug in UAE calculating the size using 32 instead of 64 bit integers.

Or it was recognised correctly and you just didn't notice it in AmigaOS 3.1 because you only used AmigaOS 3.1 tools, the old Workbench, Format, etc. are all 32 bit only and for example display 1 GB for a 5 GB partition. The size displayed by C:Info is wrong as well, but with the output of C:Info you can calculate it yourself, the number of blocks it displays is correct, if there are over 16,000,000 free blocks after formatting it's recognised correctly as 8188 MB partition. You can't use it with the AmigaOS 3.1 FFS and even formatting a 8188 MB partition may fail already, but for example formatting a 8000 MB partition should work and only destroy itself after storing several MB of data on it.

@joerg

Quote:
Actually you can use other filesystems than FFS. E-UAE can load an external filesystem handler and use it to mount a hardfile.

You specify the path to the handler to use as a parameter to the hardfile2= option (the handler binary must obviously exist somewhere in your host filesystem). If you don't specify a handler, the version of FFS built into the kickstart ROM is used.

For example:
hardfile2=rw,DH2:/home/evilrich/hardfile.hdf,32,1,2,512,0,/home/evilrich/SmartFileSystem

Cheers,
Rich