AmigaDOS is unable to allow multiple readers, but prevent multiple writers? (lock/handle)

What I want to achieve: To allow read access to a file by one or more programs, AND to allow simultaneous write access to a file by one program, but to disallow write access by several programs. i.e. Multiple readers, but no more than one writer.

I canNOT see how to do this nicely with AmigaDOS, but I'd appreciate confirmation or a solution. From past reading & observations I've found that:

* Open() with "accessMode" set to MODE_OLDFILE or MODE_READWRITE allows multiple programs to open the same file. Internally it seems to use SHARED_LOCK, even though it returns a file handle.

* Open() with "accessMode" set to MODE_NEWFILE disallows any other locks or filehandles (including read-only ones). Internally it seems to use EXCLUSIVE_LOCK, even though it returns a file handle. A limitation of MODE_NEWFILE is that it cannot be used to access an existing file (it will be deleted & a new one created in it's place).

* Lock() with "mode" set to SHARED_LOCK (aka ACCESS_READ) allows multiple program to lock the same file (as long as they also use SHARED_LOCK).

* Lock() with "mode" set to EXCLUSIVE_LOCK (aka ACCESS_WRITE) only allows one program to lock the file. The kicker is that this literally is "exclusive", so ACCESS_READ locks are disallowed too.

The above would seem to prevent obtaining a file handle with exclusive access to an *existing* file (since MODE_NEWFILE will erase it). But you can simulate what I shall call Open()'s fourth mode, by calling Lock() with EXCLUSIVE_LOCK, and then using OpenFromLock(). Kinda odd that AmigaDOS doesn't just offer MODE_EXCLUSIVE.


So I'm wondering if there is a similar kind of trick that would allow multiple readers BUT only one writer. It's fine if this trick only works with a particular program, as I'm just trying to prevent the same program being run multiple times on the same file (with write access). e.g. Multiple independant writers to a database would cause data corruption.

The only (poor) solution I can think of is that when I want to open file "X" with write access, I must first create a special second file (say "X.writeAccess") using MODE_NEWFILE, which stays Open() while I have write access. As there's an exclusive 'lock' on this special file, only one writer would be allowed. Unfortunetely this solution has several big drawbacks:

1. It's not really suitable for general-purpose file access, as it will end-up creating lots of ".writeAccess" files everywhere. And they could only be automatically deleted if the program finishes normally (rather than it or the OS crashing).

2. If a user moves/renames the main file (while the program has it open), the special ".writeAccess" file will NOT be moved/renamed, so a second writer would then be allowed.

3. AmigaDOS still allows files with exclusive locks to be move/renamed, so the user could (foolishly) move/rename the special ".writeAccess" file, and then a second writer would be allowed. (Perhaps the user doesn't understand what the ".writeAccess" file is, and thinks it's some junk they should get rid of... Never underestimate an Amiga user who knows just enough to be dangerous!)

4. It relies upon there being no *genuine* file called "X.writeAccess". Certainly it seems unlikely, but it's not impossible. And if we use a shorter special name, like say "X.lock", then it gets more likely.


One attempt to improve that (poor) solution would be to have a special folder to store all these special files, where their path mirrors that of the main path. So say the main file is "Work:Databases/VeryImportant.db" then the special file would be "Work:writeAccess/Databases/VeryImportant.db".

This has would make objection "1" less serious, as all the annoying special files would be hidden in a special folder, and completely resolve objection "4" (assuming there is no user folder called "writeAccess"). It does not solve "2" or "3" though, and has the down side of creating tons of folders (which some users might object to).


Another attempt to improve that (poor) solution would be to have a special folder to store all these special files in RAM:, where their path mirrors that of the main path. So say the main file is "Work:Databases/VeryImportant.db" then the special file would be "RAM:writeAccess/Work/Databases/VeryImportant.db".

This would entirely resolve objections "1" & "4", makes objection "3" less serious, but does not solve "2". I think an entirely different approach would be needed, if "2" was to be solved (but suspect this is impossible).

I have thought of a possible solution, but it uses AmigaDOS functionality I've never used before (and is not universally supported on AmigaOS 3.x & presumably not MOS or AROS either, nor AmigaOS 4.0 for that matter) :

LockRecord() appears to allow you to take a file handle & exclusively lock a specific part of the file. I could artibrarily lock the first byte of a file, and use this to indicate that (exclusive) write access is allowed (to the whole file).

Before a recent update of AmigaOS4.1, this feature relied upon the filingsystem to support it. Also, the SDK says that filingsystems may not support locking of regions that don't yet exist, thus the solution may not work for new (empty) files, even if the filingsystem supports record locking.

Instead of storing the special file in RAM:, one could have a 'server' running on a named Exec port, and then any requests for write access would go to this 'server' (which would store all file paths in a hash table). (If the named port did not exist, then the program would create the 'server'.)

This doesn't have much benefit over storing special files in RAM:. It would be a bit faster & more memory efficient, and would completely resolve objection "3" (rather than just partially in the case of RAM:), but it would be relatively complex to implement. And it would still fail to solve objections "2", so I'm not convinced the extra complexity would be worth it...

@ChrisH

Record Locking is supported at the DOS level from AmigaOS 4.1 FE onwards (perhaps even slitghly earlier)

@broadblues

AS to your original question of multiple reads single write, must you have continuos read access ? Normally a write is exclusive, while reading is sahared.

So how about a file notification? Then read the update file after change but release the lock after completing the read to allow further writing?

I'm not really sure what you're trying to do, but the "server" option is definitely the right approach. It knows (internally - use an Exec list, don't create temp files) what is locked and what isn't. Moreover, you probably don't even need a list - just exclusive lock everything and do all reads and writes through the server.

@ChrisH

You can use ChangeMode() to switch a lock/filehandle between shared and exclusive mode.

Changing from shared to exclusive mode will only work if there are no other locks on the file/directory in question.

@broadblues
Yes, I want "continuous" read access. The reason is a bit convoluted, and as with most of my "strange" coding needs is due to having a general-purpose OS-independant (portable) abstraction.

Quote:
AFAIK, on AmigaOS write access is NOT exclusive, since internally MODE_READWRITE is just a shared lock. So if you have a file handle, then you can always read or write. There isn't even MODE_EXCLUSIVE, so you must kludge it via Lock()+OpenFromLock().

I quite like that on AmigaOS, when one program has a file open for writing, I can still read it, back it up, etc. It just seems a bit strange (and potentially dangerous) to allow multiple writers.

@Chris
Reading/writing to a file THROUGH a 'server' doesn't seem especially efficient (2 extra context switches per I/O operation+other overheads). And would perform poorly once several files were being simulataneously accessed from different programs (when the server is performing an I/O operation for one program, it would block all other programs.) Solving that would mean creating a new process for each file being accessed, which starts to look very complex & not at all efficient.

@ChrisH
Quote:
Reading a file while another is writing to it sounds equally dangerous to me. Isn't there a possibility of reading corrupt data? For example, if the writing program writes a buffer that ends with half of a file path and the reading program reads the file before the next buffer is written, won't a corrupt file path be read??

No, because the filesystem is responsible for making sure that what is read from the medium is the lastest, most up-to-date version.

So either the Read will be taken from the cache or any outstanding Writes will be flushed before the Read is performed.

@xenic Quote:
I am mainly thinking of it in a backup scenario:

On Windows you need special handling to deal with files which are opened for writing, even if no actual writing is taking place at the time.

On the Amiga you can just backup everything using normal copy routines. If you make sure that no programs are actively doing anything, then it's usually pretty safe to do a backup.


Another benefit, mainly due to the Amiga's lack of resource tracking, is that if a program crashes with a file open, then you can still read that file.


The one case where the Amiga doesn't let you do these things is when a program is creating a new file. In that case it is (internally) exclusively locked. That makes a lot of sense too (it's fairly guaranteed to be a half-finished file, so no point in backing it up or reading it).