| |
ADF floppy disc image reader |
| |
This is a long thread. Click here to view the threaded list. |
| |
|
Tom Walker |
Message #120136, posted by TomWalker at 11:31, 19/4/2012, in reply to message #120134 |
Member
Posts: 25
|
Don't suppose you remember or can find out the timing for this? It's simply 'real time', ie clocking the MFM data at 500 kbit/s.
EDIT: I should add, I took a quick look though the source for 0.8 but couldn't see anything related. Not surprising as APD support didn't turn up until 0.99.
PS: Are you going to fix the read track command and put out 1.0?  I'm trying to pass a degree here... and there is a lot of other work needed before v1.0 is released. |
| |
 [ Log in to reply ] |
| |
|
Trevor Johnson |
Message #120137, posted by trevj at 11:51, 19/4/2012, in reply to message #120136 |
Member
Posts: 660
|
I'm trying to pass a degree here Have you considered blacklisting a few sites for a month or so? |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120139, posted by sirbod at 12:29, 19/4/2012, in reply to message #120136 |
Member
Posts: 563
|
It's simply 'real time', ie clocking the MFM data at 500 kbit/s. I'll store the sector timing in ms (200ms being one complete track rotation) as I decode the data during loading, then advance the head to the relevant sector based on the density being requested and time mod 200ms since the last DiscOp.
If my calculations are correct and I'm understanding the 1772 correctly, a 4MHz ARM will always skip 1 whole sector between DiscOp's but a StrongARM won't - so I'll have to hardcode a minimum skip of 1 whole sector to emulate a 1772 on an original Arc. I doubt there's any protection that's StrongARM dependent, I can always make it configurable should it be needed.
Thank you for that info. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120153, posted by sirbod at 22:30, 19/4/2012, in reply to message #119561 |
Member
Posts: 563
|
1.24 available, which...wait for it...drum roll...all APD's I have now work.
Thanks to Tom's help, we now know Fire & Ice is missing two quad density tracks and needs re-imaging before it will work.
There are still bugs to fix, such as changing disks via the keys randomly crashes.
To get around the dreaded "Bad address offset", I spent several hours moving data / variables around and making use of ADRL / LDRL / STRL - then spent a few more hours fixing random crashes they introduced.
For the list of known working APD titles and patches for newer kit, see message 119907 above.
Changes from 1.22
- Fixed a bug in MemCopy that only occured when the source address was +1 off word alignment and the destination was word aligned. This was causing Sensible Soccer to fail.
- Started implementing the head advancing code, there's enough to fix Sensible Soccer. It now calculates the time of each sector from the start of the track, so it can position the head on the next DiscOp accordingly.
[Edited by sirbod at 23:31, 19/4/2012] |
| |
[ Log in to reply ] |
| |
|
Michael Drake |
Message #120165, posted by tlsa at 09:47, 20/4/2012, in reply to message #120153 |
Posts: 1097
|
Great! Thanks for doing all this. It makes running old games much easier. Also the multidisc support is great for ArcEm, because the RISC OS ArcEm front end doesn't support changing floppy discs.
Edit: Attached a !Sprites file for ADFFS with rectangular pixel format.
[Edited by tlsa at 12:15, 20/4/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120167, posted by sirbod at 11:45, 20/4/2012, in reply to message #120165 |
Member
Posts: 563
|
Attached a !Sprites file for ADFFS with rectangular pixel format. Cheers, I was looking at fixing that late last night. I'll put it in the next release. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120181, posted by sirbod at 13:59, 23/4/2012, in reply to message #119561 |
Member
Posts: 563
|
1.25 available, this fixes the random crashing when swapping discs via keys.
Known issues
- Writing to a mounted adf is not working when RISC OS is in byte mode (ie on RISC OS 3.1)
- The Filer crashes when exiting via the menu option
[Edited by sirbod at 17:29, 24/4/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120187, posted by sirbod at 17:59, 24/4/2012, in reply to message #119561 |
Member
Posts: 563
|
1.26 available
Changes since 1.25
- Writes to ADF when ADFS is in byte mode work
- An addition check for Ctrl-Shift-F? has been added to MiscOp 1 (poll change), to get around games that hijack the insV vector
- If a disc request has been pending for 40 disc polls, MiscOp 1 now forces the disc change manually. This is to get around games that hammer MiscOp 1 looking for a disc, leaving no time for the OS to handle transient callbacks. This won't work on physical kit, I'm working on a fix for that.
- Filer no longer crashes when exited via the menu
Known issues
- Wolfenstein hasn't worked since 1.24, I've yet to write the time based head advance code to fix this and Sensible Soccer. SS currently works however. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120252, posted by sirbod at 22:10, 30/4/2012, in reply to message #119561 |
Member
Posts: 563
|
1.28 available
Changes since 1.26
- Mounts JFD images files - Jon's Floppy Disc Image or JFDI for short 
- Virtual disc head is now advanced based on time, so both Sensible Soccer and Wolfenstein work
- DiscOp recording added, in preparation for floppy imaging
- Fire & Ice has crashed since 1.26, now fixed
- Now generates "new error style" on newer OS's, although I can't test it at the minute
Known issues
- Darkwood hangs when changing to disc 6, sure I had it working at one point as I laughed in amazement at the bifuricating disc swaps!
- If an error occurs during a background transfer, it hangs the machine. Disc 1 and 6 of Simon the Sorcerer (which are truncated ADF images) do this when copying to the harddisk |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120306, posted by sirbod at 19:35, 9/5/2012, in reply to message #119561 |
Member
Posts: 563
|
1.30 available
Changes since 1.28
- Lots of bug fixes
I've spend over a week trying to fix the hanging during background transfers, when a disc error occurs - to no avail. I've tried mirroring the ADFS source code - which still hangs.
The problem is actually FileCore hanging, I suspect its down to it getting stuck in a loop processing the scatter list result and possibly a bug. Ironically, if you perform a copy in the Filer, the disc error is reported. Clicking on a button then causes FileCore to hang.
I've noticed another interesting fact, it looks like the ADFS Filer in RISC OS 3.7 (and possibly newer) doesn't fill the scatter list completely when files are being copied. It watches the entries and only adds more as they go empty - very nasty. I can understand it watching the scatter list, to report how many bytes are remaining, but leaving a partial terminated list is a bit naughty. I expect it was done to handle very large files - I'm amazed copies work in the Filer under ADFFS, I'd expect them to terminate early due to the speed.
The irony is...whilst Filer is showing the copy dialog - it slows the machine down so much whilst updating it, it may as well be a foreground copy!
Known issues
- Darkwood hangs when changing to disc 6, sure I had it working at one point as I laughed in amazement at the bifuricating disc swaps!
- If an error occurs during a background transfer, it hangs the machine. Disc 1 and 6 of Simon the Sorcerer (which are truncated ADF images) do this when copying to the harddisk |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #120307, posted by Phlamethrower at 20:23, 9/5/2012, in reply to message #120306 |
 Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
I've noticed another interesting fact, it looks like the ADFS Filer in RISC OS 3.7 (and possibly newer) doesn't fill the scatter list completely when files are being copied. It watches the entries and only adds more as they go empty - very nasty. This is a documented feature of FileCore - see the docs on background transfers in the PRM (2-603). I think you can stop FileCore from extending the list by clearing bit 30 in the process status word. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120312, posted by sirbod at 22:30, 9/5/2012, in reply to message #120307 |
Member
Posts: 563
|
This is a documented feature of FileCore - see the docs on background transfers in the PRM (2-603). I think you can stop FileCore from extending the list by clearing bit 30 in the process status word. Don't suppose you know which file the FileCore source code is in, that handles the background transfers? I need to debug it to find out why it locks the machine.
I'm setting the status flags to 0 on entry, to prevent extensions to background transfers by the way. |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #120313, posted by Phlamethrower at 23:45, 9/5/2012, in reply to message #120312 |
Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
It looks like s.FileCore80 contains most/all of the code that deals with background transfers. Search for ProcessStatus, CanExtend, etc. (all defined near the end of s.Defns). |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120317, posted by sirbod at 08:48, 10/5/2012, in reply to message #120313 |
Member
Posts: 563
|
Thanks, I'll take a good look through it today.
EDIT: Yay, fixed. Undocumented feature...if an error occurs during the foreground part of a background transfer, you need to return the error in both the foreground (set V / R0 is error on exit) and the background (store the error / pointer to scatter entry that caused it, in the scatter list status words)
[Edited by sirbod at 11:13, 10/5/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120340, posted by sirbod at 16:05, 10/5/2012, in reply to message #119561 |
Member
Posts: 563
|
1.31 available
Apart from the Darkwood issue (which I've yet to investigate), this is feature complete and will be released as 2.0 with source soon.
I'm unsure if the software vector tracking added in this version will work on RISC OS past 3.7 - swapping disc via hotkeys will crash if it's not working. The sure test is "does Cataclysm work"
If people on later OS versions could test and report back, that would be great.
Should it fail, you can turn the feature off in !Boot.
NOTE: It takes a copy of the software vector table as it loads, if any of the key vectors change it only resets them during a hotkey disc change.
Changes since 1.28
- No longer hangs if a disc error occurs during a background transfer (thanks go to Jeffrey for pointing me in the right direction)
- Now tracks changes to FileV, ArgsV, BPutV, GBPBV, FindV and resets whilst changing disc in-game via the hot keys. This fixes games that use privatisation modules (eg Cataclysm).
Known issues
- Darkwood hangs when changing to disc 6 |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120399, posted by sirbod at 19:03, 15/5/2012, in reply to message #120340 |
Member
Posts: 563
|
1.33 available
Changes since 1.31
- Type D / L floppies mount correctly
- Fixes for disc image recording
Known issues
- Darkwood hangs when changing to disc 6
- Gribbly's Day Out disc protection failing
[Edited by sirbod at 14:25, 21/5/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120474, posted by sirbod at 16:07, 23/5/2012, in reply to message #119561 |
Member
Posts: 563
|
1.34 available
Changes since 1.33
- Gribbly's Day Out is working again
- Lots of bug fixes, including fixes to disc errors returned, RISC OS recognising disc changes in the GUI, Flush not working in the Filer to name a few big ones.
- Shift-Break to boot floppies is now supported (for the time being, hit F12, type "ADFS" / "DRIVE 0" and then do a Shift-Break to boot from the mounted floppy)
Known issues
- Darkwood hangs when changing disc
- Caps Lock turns on when cataloguing a mounted floppy (fixed in next release)
[Edited by sirbod at 18:22, 23/5/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120499, posted by sirbod at 21:00, 28/5/2012, in reply to message #119561 |
Member
Posts: 563
|
1.35 available
Changes since 1.34
- CAPS lock no longer being turned on when swapping discs
- Mounting a JFD would allocate twice the amount of track/sector table memory required (~8k)
- Dismounting a disc was randomly crashing Filecore
Known issues
- Darkwood hangs when changing disc (now confirmed to not be an issue with ADFFS)
- RMTidy kills the ADFFS filer and dismounts the current image
- RMA space creeps up by exactly 64kb when swapping disc via hotkeys...on roughly the 9th change. I've confirmed it's not ADFFS leaking memory, doesn't look like ZLib or CLib either.
[Edited by sirbod at 13:04, 29/5/2012] |
| |
[ Log in to reply ] |
| |
|
Jason Togneri |
Message #120500, posted by filecore at 04:34, 29/5/2012, in reply to message #120499 |
Posts: 3868
|
RMA space creeps up by exactly 64kb when swapping disc via hotkeys...on roughly the 9th change. I've confirmed it's not ADFFS leaking memory, doesn't look like ZLib or CLib either.  |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120502, posted by sirbod at 08:04, 30/5/2012, in reply to message #120500 |
Member
Posts: 563
|
The RMA creep looks like a feature of the heap manager.
SharedCLibrary is claiming several blocks, which it releases shortly after. Although they've been freed, they're still showing in the heap as allocated:
RMA Claim - SharedCLibrary - 2000 - result 224AB54
RMA Claim - SharedCLibrary - 4000 - result 22600F4
RMA Claim - SharedCLibrary - 1BCC - result 2248F84
RMA Claim - SharedCLibrary - 1000 - result 2264104
RMA Claim - SharedCLibrary - 8000 - result 2265144
RMA Claim - ADFFS - C7F04 - result 226D124
RMA Claim - ADFFS - 1E40 - result 2335034
RMA Free - SharedCLibrary - 8000 - 2265144
RMA Free - SharedCLibrary - 1BCC - 2248F84
RMA Free - SharedCLibrary - 4000 - 22600F4
RMA Free - SharedCLibrary - 2000 - 224AB54
RMA Free - SharedCLibrary - 1000 - 2264104
...then a similar sequence on the next disc load, only difference being ADFFS releases it's two blocks before the next load
The tail end of the RMA heap after this is:
2248F84 (1BD0) - Alloc (should be Free)
224AB54 (2010) - Alloc (should be Free)
224CB64 (1010) - Alloc
224DB74 (1BD0) - Alloc
224F744 (4010) - Alloc
2253754 (2010) - Alloc
2255764 (2980) - Alloc
22580E4 (8010) - Alloc
22600F4 (4010) - Alloc (should be Free)
2264104 (1010) - Alloc (should be Free)
2265144 (8010) - Alloc (should be Free)
226D124 (10C10) - Free
227DD34 (C7F10) - Alloc
2345C44 (1E50) - Alloc
As you can see, the five blocks claimed and released by SharedCLibrary are all still showing as Alloc in the heap. They total up to 66K, which leaks every time this happens.
I've confirmed this happens on RO 3.10 and 3.70, no idea about later RO versions.
Anyone have any ideas?
[Edited by sirbod at 02:36, 28/11/2012] |
| |
[ Log in to reply ] |
| |
|
Andrew Rawnsley |
Message #120503, posted by arawnsley at 09:24, 30/5/2012, in reply to message #120502 |
R-Comp chap
Posts: 600
|
I wonder if this is anything to do with a problem we saw on ARMini a few months ago, that was ultimately fixed quite recently in OS5 (but was probably there for ages)...
In our case, we had an app which would periodically poll SCSI:1 and SCSI:2 to see if a new disc had been inserted, so that it could be auto-mounted and the iconbar icons updated accordingly with the disc name and appropriate device icon.
99 times out of 100, the drive would be empty, so there would be a silent "disc not found" type error message generated. These errors would stay in the heap, which would gradually expand.
After a few days of operation, the heap would reach 1Mb+ and odd crashes would occur.
I believe this only got fixed in March/April, but had been around for ages. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120507, posted by sirbod at 14:51, 30/5/2012, in reply to message #120503 |
Member
Posts: 563
|
Could be, I've only managed to produce it when the code is called under something interrupt driven (transient callback or InsV)
It's odd it only happens with CLib RMA releases, the ADFFS allocations release fine - I'd expect them to suffer the same problem. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120508, posted by sirbod at 21:10, 30/5/2012, in reply to message #120507 |
Member
Posts: 563
|
I've had a thought...does OS_Module Free do any kind of validation checks when it's called? If so, could it be failing and returning before releasing the memory?
Could CLib be calling it in the wrong CPU mode (I believe it only uses RMA when in Supervisor mode)
Where's the source code for CLib and OS_Module?
Does anyone know what the highest version of CLib is that will run on 3.10? And where I can get it?
I'm guessing the answers to all my questions are on RISC OS Open somewhere! |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120511, posted by sirbod at 13:02, 31/5/2012, in reply to message #120502 |
Member
Posts: 563
|
Recoding the disc change to happen during an SWI vector event has stopped the memory leak, so its looking like its related to callbacks in my case.
I have however noticed another quirk with the heap manager, if I request 7K in the RMA and there are several free blocks slightly larger (13K, 12K, 48K), instead of using one of the smaller blocks, it allocates memory from the end of the RMA.
One of the biggest uses of the RMA is WindowManager (on RO 3.7 at least), it claims and releases dozens of small blocks which can't be very efficient. If you have a Task window open and type, it goes crazy allocating and freeing 52 bytes for every key press - is it worth it? I think it needs re-writing to use its own scratch space - its no wonder Task windows are so slow!
[Edited by sirbod at 14:03, 31/5/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120513, posted by sirbod at 19:11, 31/5/2012, in reply to message #119561 |
Member
Posts: 563
|
1.36 available
Changes since 1.35
- Disc changes via hot keys are now actioned during an SWI event, instead of via a callback. This is to work around the RMA memory leak caused when CLib tries to release its allocations.
Known issues
- RMTidy kills the ADFFS filer and dismounts the current image
- Sensible Soccer hangs up when changing disc (new bug in 1.36)
- SilverBall is not recognising disk changes
[Edited by sirbod at 23:13, 1/6/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120572, posted by sirbod at 20:01, 12/6/2012, in reply to message #119561 |
Member
Posts: 563
|
1.37 available
Changes since 1.36
- ADFS Dismount no longer ejects the floppy disc, use ADFEject or Eject from the Filer to eject and release the memory. This is to fix games that eject discs between disc changes (eg Silver Ball)
- Removes ZLib from memory when you quit the FS
- OS_Module 8/9 (RMTidy / RMClear) are now ignored whilst ADFFS is loaded
- OS_Byte 201 (disable keyboard) is now forced to enable the keyboard. This it to fix games that disable the keyboard at disc change prompts (eg, Oh, No! More Lemmings)
- Hotkey disc changes now work after a soft-reset (broke in 1.36)
Known issues
- Filer doesn't reinitialise correctly after a soft-reset
- Sensible Soccer hangs up when changing disc
- Games that check if the disc is write protected currently don't work
- Games that check the read speed of files currently don't work (eg Technodream, see potential fixes section for a workaround) |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120681, posted by sirbod at 21:01, 22/6/2012, in reply to message #119561 |
Member
Posts: 563
|
1.39 available
Had a fairly heavy rewrite of the disc change code, to fix issues caused since switching from CallBacks, to code sitting on the SWI vector.
Also added a simulated Shift-Break reset - select "Boot floppy" from the Filer to boot floppies that will only run via shift-break.
Also includes various converters, APDtoADF / APDtoJFD and ConvRec to convert recordings into JFD files.
If you wish to image any protected floppies, use the following method:
1. Clean boot the machine if disc 1 has been seen previously
2. Load ADFFS
3. Start recording via: *ADFRecord adfs::4.$.recording
4. Insert disc 1
5. Copy all files from it to the RAM drive or a temp folder (this is to ensure all sectors with files on are noted)
6. Launch the game, once it's past the disc protection check, or prompts for disc 2, hit Ctrl-Shift-F10 to stop the recording
7. Modify the variables in the header of ConvRec and run it with disc 1 in the drive
To image disc 2 onward (which usually aren't protected):
1. Clean boot the machine if the disc has been seen previously
2. Insert disc X
3. *ADFRecord adfs::4.$.recording
4. Copy all files to the RAM drive or temp directory
5. *ADFStop or hit Ctrl-Shift-F10 to stop recording
6. Modify the variables in the header of ConvRec and run it with disc X in the drive
As I've yet to write the delta write code for JFD's; save game discs are probably best imaged as ADF (using Dimager or ADFimager), so they can be flushed back to the HD.
Changes since 1.37
- Sensible Soccer now changing discs correctly
- Heimdall now changing discs correctly
- Boot floppy command and option added to the Filer
- Now includes ConvRec to convert recordings into JFD files
- Displays information on JFD files under Format/Current Format
Known issues
- Filer doesn't reinitialise correctly after a soft-reset
- Games that check if the disc is write protected currently don't work
- Games that check the read speed of files currently don't work (eg Technodream, see potential fixes section for a workaround)
- Chuck Rock not detecting disc changes correctly
- DarkWood hangs after changing discs
- Fire & Ice fails the protection check, if not run from the Harddisc
[Edited by sirbod at 23:47, 23/6/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120746, posted by sirbod at 14:35, 2/7/2012, in reply to message #119561 |
Member
Posts: 563
|
2.00 available
Source code included.
Changes since 1.39
- Filer now shuts down and start up correctly when the desktop is started/quits
Known issues
- Games that check if the disc is write protected currently don't work
- Games that check the read speed of files currently don't work (eg Technodream, see potential fixes section for a workaround)
- Fire & Ice fails the protection check, if not run from the Harddisc
- After booting from a floppy via the Filer, it doesn't reset "Boot" after the floppy has booted
[Edited by sirbod at 18:52, 4/7/2012] |
| |
[ Log in to reply ] |
| |
|
Philip Webster |
Message #120747, posted by pwx at 14:54, 2/7/2012, in reply to message #120746 |
Member
Posts: 227
|
2.00 available
Is there an archive of earlier versions? |
| |
[ Log in to reply ] |
| |
|
Trevor Johnson |
Message #120749, posted by trevj at 15:08, 2/7/2012, in reply to message #120746 |
Member
Posts: 660
|
2.00 Congratulations!  |
| |
[ Log in to reply ] |
| |
| Pages (8): |< <
6
> >|
|
