The Icon Bar: General: ADF floppy disc image reader
|
ADF floppy disc image reader |
|
This is a long thread. Click here to view the threaded list. |
|
Jon Abbott |
Message #120055, posted by sirbod at 08:06, 11/4/2012, in reply to message #120052 |
Member
Posts: 563
|
I now have Zarch loading on 1.20, debugging the code it tries to write 0's from &1FED400 to &2000000 - which doesn't exist. Eep!
"Pass me a pickaxe..." You'll have to explain...not had much sleep lately!
EDIT: Thanks for confirming my suspicion Jeffrey. I'll see if it can be patched somehow.
[Edited by sirbod at 09:07, 11/4/2012] |
|
[ Log in to reply ] |
|
Steffen Huber |
Message #120058, posted by hubersn at 15:25, 11/4/2012, in reply to message #120053 |
Member
Posts: 91
|
if you were to go for option 3, would be to use a pre-existing format - the extended DSK format, as used by Speccy and CPC stuff, seems a reasonable alternative. Seems to cover the mixed density okay (although will still need pre-processing in my case), but still leaves us with no multi-disc set support, or patching for older games.
On second thoughts I don't think DSK will work for discs using the R.A.North protection, which has mixed SD and DD sectors within Track 40 Head 0. I suppose you could put two Track 40's in, but it would break the Track Offset table. Take the following with a pinch of salt - I had a CPC once and was involved in the development of a few CPC emulators, but I have forgotten a lot of things and probably never knew many other things.
EDSK format was specified for CPC and Spectrum usage, both of which use a NEC FDC 765. The 1772 is backwards compatible with the 765, but adds quite some features. Apart from the Archimedes, the Atari ST also used the 1772. So it is unlikely that EDSK would allow a faithful image of all discs that could be read by the 1772. I am very sure that there were Atari ST discs that could not be duplicated with even the most sophisticated CPC copy software.
So it would be sensible to check out which disc image formats were invented for the Atari ST, as it is likely the best match for our needs. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120059, posted by sirbod at 16:06, 11/4/2012, in reply to message #120058 |
Member
Posts: 563
|
EDSK format was specified for CPC and Spectrum usage, both of which use a NEC FDC 765. The 1772 is backwards compatible with the 765, but adds quite some features. Apart from the Archimedes, the Atari ST also used the 1772. So it is unlikely that EDSK would allow a faithful image of all discs that could be read by the 1772. I am very sure that there were Atari ST discs that could not be duplicated with even the most sophisticated CPC copy software. Come to think of it, I can see it struggling with "sector within sector" as well as the mixed density track I've already mentioned. Wolfenstein for example has a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector within a sector - that's a lot of nested sectors!!! |
|
[ Log in to reply ] |
|
Martin Bazley |
Message #120060, posted by swirlythingy at 17:35, 11/4/2012, in reply to message #120059 |
Posts: 460
|
Do forgive me if I'm being thick, but is there anything stopping you simply reading the raw bits off the disk, one at a time, dumping them to a file, and then emulating in software whatever it is the floppy controller does in hardware?
I mean, if all these obscure sector tricks and all information necessary to determine the number of sectors/density of disk can be represented in the confines of the ADFS disk format, and you know the ADFS disk format, then there's no problem, surely? Are you trying to effectively reformat all your floppies while preserving the data? |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120061, posted by sirbod at 18:49, 11/4/2012, in reply to message #120060 |
Member
Posts: 563
|
Do forgive me if I'm being thick Anyone seen Martin's coat?
That's what we're currently doing. The problem is, you can't easily duplicate exactly what's on a floppy unless you already know it's geometry...hence why protected discs aren't easy to copy - you end up with a compromise.
APD is more than adequate, the problem is that as you don't know the geometry, you end up with a larger image than would be on the disk. Not an issue for an emulator with lots of memory, but an issue for ADFFS when running on physical machines with 4mb of RAM. I'm trying to keep the memory footprint down to 1mb, to expand the tin it will run on.
I started this particular discussion because, as ADFFS is sitting between the software and the hardware, it knows the geometry of the disc from the disc reads - making it ideal to image discs natively on an Arc. I just need to record the data read off the disc, along with the track/sector/head/density of the initial request and I have the disc imaged...obviously there's a bit more to that, as not every part of the disc is touched straight away.
However...I can't create an APD from that easily as APD isn't sector aware - 99.9% of disc op's are sector based, not track based. Protection methods such as mixed density and sector within sector may not work all the time.
So...long story short, if I want to extend ADFFS to allow image creation, I need a new image format, that is sector aware.
In addition to these issues, I'd like to provide a single solution for multi-disc games, game instructions, cover art etc and automated patching so the older games run on newer kit. The idea being "software preservation" and not "floppy disc preservation" as other formats aim to be. I'm not really fussed about preserving a floppy disc, just the software on it and any protection sectors.
You could argue about the patching bit - but I think if the software is kept vanilla and patches are applied via delta/script based on hardware/OS spec it shouldn't be a problem and will allow them to be patched in the future for newer hardware/OS's as they come out.
Think MAMEUI for the Arc world
[Edited by sirbod at 19:52, 11/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120062, posted by sirbod at 19:32, 11/4/2012, in reply to message #120021 |
Member
Posts: 563
|
Latest update...
I have mixed density support working, I'm just re-writing DiscOp 2 to fix writes before I release.
There's also a problem with sequenced disc requests (single sided discs), which I just can't figure out...I'll explain with the two examples I know of:
1. Zarch requests single density, Track 40, sector 240 / 241 - the data in the APD is actually in Track 80, sector 240 / 241
2. Drop Ship requests double density, Track 159, sector 1 - the data in the APD is in Track 159, sector 1
I'd expect to have to double, halve or leave the track number for a sequenced disc request, depending on how the image is created. Tracks on a 1772 physical floppy are alternated side to side. ie Track 0 is on side 0, Track 1 is on side 1, Track 2 is on side 0 etc.
Zarch does have a CRC error on track 40, right where I'd expect sectors 240/241 to be - but there's no single density data there. Could the APD be wrong? Seems unlikely if it works under Arculator.
The Drop Ship request also has me stumped, as it's outside a single sided disc - there are only 80 tracks on a single sided disc and 160 on a double sided. Technically, it's an illegal request and would end up reading the last track on the second side. I think it's only working, because ADFS isn't validating it.
Anyone have any insight into this?
For the time being I've botched it, but would to resolve it correctly. |
|
[ Log in to reply ] |
|
Martin Bazley |
Message #120063, posted by swirlythingy at 22:56, 11/4/2012, in reply to message #120062 |
Posts: 460
|
The Drop Ship request also has me stumped, as it's outside a single sided disc - there are only 80 tracks on a single sided disc and 160 on a double sided. Technically, it's an illegal request and would end up reading the last track on the second side. I think it's only working, because ADFS isn't validating it. Sounds like exactly the kind of undefined ADFS behaviour which old software would rely on, and hence needs to be emulated. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120067, posted by sirbod at 09:45, 12/4/2012, in reply to message #119873 |
Member
Posts: 563
|
Although I fixed the scatter list on reads, writes now lock the machine solid Think I figured out scatter lists.
DiscOp 1 - Read Foreground request - Read until scatter list terminates (the pair is <-65536, 0) Background request (R1 bit 8 set - &1xx) - Read until scatter list terminates (ie the pair is [<-65536, 0])
DiscOp 2 - Write Foreground request - Write until R4 (bytes requested) is 0 Background request (R1 bit 8 set) - Write until scatter list terminates (ie the pair is [<-65536, 0])
That doesn't seem to match the scant documentation, but performs read/writes without hanging. Needs thorough testing though before I update the RISC OS OPEN documentation.
[Edited by sirbod at 19:01, 12/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120068, posted by sirbod at 15:38, 12/4/2012, in reply to message #119561 |
Member
Posts: 563
|
I've started putting tasks into the first post, should anyone want to help expand this project.
I'll add to it as and when I come across issues with discs. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120069, posted by sirbod at 17:18, 12/4/2012, in reply to message #119561 |
Member
Posts: 563
|
1.21 available, which adds mixed density support.
Magic Pockets and Sensible Soccer are working, although Zarch is hanging. Although it was initially working, changes made for Magic Pockets / Sensible Soccer may have broken it.
The 1772 emulator has had a complete overhaul to support mixed FM / MFM, there is the potential that some discs may have regressed. I've done limited testing and so far have only found a problem with Zarch. Single Sided discs are implemented, but will almost certainly break on most discs. This is still work in progress and for the time being is botched, so that Drop Ship, Zarch, Sensible Soccer and Magic Pockets get past the protection checks.
For the list of known working APD titles and patches for newer kit, see message 119907 above.
Changes from 1.19 - Single/Quad density support added - Writing to disc fixed, including files over one sector in length - MemAlloc module added for !Run patching - Flush and Save As now report errors from the command line, if an APD is mounted
[Edited by sirbod at 23:49, 12/4/2012] |
|
[ Log in to reply ] |
|
Michael Drake |
Message #120086, posted by tlsa at 15:26, 15/4/2012, in reply to message #120069 |
Posts: 1097
|
1.21 available, which adds mixed density support. Thanks! It's really coming along. I noticed that the ChangeLog file mentions up to version 1.20 only, but it seems to have all the changes for 1.21 anyway.
For the list of known working APD titles and patches for newer kit, see message 119907 above. That's quite impressive. About the "game specific issues", are they things that you only need to do when you're using certain processor or OS combinations?
[Edited by tlsa at 16:26, 15/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120088, posted by sirbod at 17:07, 15/4/2012, in reply to message #120086 |
Member
Posts: 563
|
Thanks! It's really coming along. I noticed that the ChangeLog file mentions up to version 1.20 only, but it seems to have all the changes for 1.21 anyway. I forgot to update the file before posting!
About the "game specific issues", are they things that you only need to do when you're using certain processor or OS combinations? Those are changes required for RISC OS 3.7 - I'll add that to the section.
I'm just coding the multi-disc handler and should have a new version available tomorrow. I then need to fix Zarch and it's more or less complete, pending bug reports and failing protection methods on discs I've yet to analyse.
[Edited by sirbod at 18:43, 15/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120107, posted by sirbod at 13:41, 17/4/2012, in reply to message #119561 |
Member
Posts: 563
|
1.22 available, which adds multi-disc support and fixes Zarch.
As usual, there are some multi-disc specific caveats:
1. Image files must be identical in name, with the disc number as the last character of the filename eg.
FFT Disc 1 FFT Disc 2 2. The key combination is currently Ctrl-Shift-F1 thru F9 - which will change at some point as I think it conflicts with the normal F-key aliasing
3. It crashes horribly on physical machines, but seems to work under emulators.
4. If it fails to allocate enough module space for the new disk image, it will fail silently and eject the current image.
For the list of known working APD titles and patches for RISC OS versions, see message 119907 above. |
|
[ Log in to reply ] |
|
Tom Walker |
Message #120111, posted by TomWalker at 17:55, 17/4/2012, in reply to message #120107 |
Member
Posts: 25
|
For the list of known working APD titles and patches for RISC OS versions, see message 119907 above. I notice that Fire & Ice is on this list. Does the protection fully work on this? The effects won't kick in until level 3 (which will have no enemies on it). I never got it fully working on Arculator. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120112, posted by sirbod at 18:28, 17/4/2012, in reply to message #120111 |
Member
Posts: 563
|
Haven't tested beyond the game loading, although have made a note to see what it's doing with level 3.
I need a break from coding, so perhaps I'll give it a go tonight.
EDIT: Level 3 is devoid of life. I'll analyse it but suspect we may need to re-image the original floppy. Anyone have the original floppy if we need it? I'll knock up rudimentary capture code for ADFFS if required.
[Edited by sirbod at 21:12, 17/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120113, posted by sirbod at 23:34, 17/4/2012, in reply to message #120111 |
Member
Posts: 563
|
I've analysed Fire & Ice and recorded what it does three times. I suspect it may have a bug in the code as it randomly tries to read data on disc 2 with illegal DiscOp 3 calls (read track). It did this once in the three records playing up to level 3.
The only protection check on disc 1 is a DiscOp 3 to track 158 - which in my case will return nothing as I haven't coded it yet! It should work on Arculator though - provided the track is correct in the APD.
If someone has the original disc 1, send me the output of the following code:
DIM data 10240 FOR a%=0 TO 10239 STEP 4:data!a% = 0:NEXT DIM z% 64 FOR a%=0 TO 63 STEP 4:z%!a% = 0:NEXT z%?0=10 z%?1=5 z%?2=2 z%?3=2 z%!16=&D00C00 SYS "ADFS_DiscOp", 0, 3+(z%<<6), &316<<(z%?0), data, 0 TO a%, b%, c% PRINT "R2=";~c% OSCLI "MEMORY "+STR$~data+" +1400" EDIT: Looks like the problem on Arculator is what's being returned by a DiscOp 3. Instead of returning the list of sector ID's, it's returning the whole track with decoded FM sector headers and all!
Maybe I'm misunderstanding DiscOp 3, but the wording implies it should return all the track data for a floppy, and only return the ID list if R1 bit 4 is set. However if I test it on physical tin it only ever returns the sector ID's.
I'll try coding this into ADFFS and see if it makes any difference.
[Edited by sirbod at 16:25, 18/4/2012] |
|
[ Log in to reply ] |
|
Tom Walker |
Message #120114, posted by TomWalker at 13:04, 18/4/2012, in reply to message #120113 |
Member
Posts: 25
|
00009014 : 0102004F 0103004F 0104004F 0105004F : O...O...O...O... 00009024 : 0106004F 0107004F 0108004F 0109004F : O...O...O...O... 00009034 : 010A004F 010B004F 010C004F 010D004F : O...O...O...O... 00009044 : 010E004F 010F004F 0110004F 0111004F : O...O...O...O... 00009054 : 0112004F 0113004F 0101004F 00000000 : O...O...O.......
On the RiscPC anyway. R260 gave disc error 04.
[Edited by TomWalker at 14:05, 18/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120115, posted by sirbod at 14:09, 18/4/2012, in reply to message #120114 |
Member
Posts: 563
|
I'm now getting those values...fingers crossed.
EDIT: It's getting further, as after the DiscOp 3 it now does a DiscOp 0 (verify) on track 159 sector 0 with the same disc parameters.
I suspect if you fix the 1772 read track code in Arculator it may well work. Having said that, I still see a blank level 3, but expect that's my DiscOp 0 code returning the wrong value in one of the registers.
See what the following returns:
DIM data 10240 FOR a%=0 TO 10239 STEP 4:data!a% = 0:NEXT DIM z% 64 FOR a%=0 TO 63 STEP 4:z%!a% = 0:NEXT z%?0=10 z%?1=5 z%?2=2 z%?3=2 z%!16=&D00C00 SYS "ADFS_DiscOp", 0, 0+(z%<<6), &31B<<(z%?0), data, &400 TO a%, b%, c%, d%, e% PRINT "R2=";~c%;" R4=";e% OSCLI "MEMORY "+STR$~data+" +400"
I expect it to error (need to know what error) Then change "ADFS_DiscOp" to "XADFS_DiscOp" which I expect to return something. I need that and the values for R2 and R4
[Edited by sirbod at 16:25, 18/4/2012] |
|
[ Log in to reply ] |
|
Jeffrey Lee |
Message #120116, posted by Phlamethrower at 15:13, 18/4/2012, in reply to message #120115 |
Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
I'm suprised either of those bits of code work at all. Look at the first two lines
DIM data 10240 FOR a%=0 TO 10239 STEP 4:z%!a% = 0:NEXT |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120117, posted by sirbod at 15:26, 18/4/2012, in reply to message #120116 |
Member
Posts: 563
|
I'm suprised either of those bits of code work at all. Look at the first two lines
DIM data 10240 FOR a%=0 TO 10239 STEP 4:z%!a% = 0:NEXT Corrected - just for you! |
|
[ Log in to reply ] |
|
Tom Walker |
Message #120118, posted by TomWalker at 16:25, 18/4/2012, in reply to message #120115 |
Member
Posts: 25
|
With ADFS_DiscOp :
Disc error 23 at :0/000C5800 at line 100
With XADFS_DiscOp :
R2=C5800 R4=1024
Resulting data is all zeroes. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120119, posted by sirbod at 16:33, 18/4/2012, in reply to message #120118 |
Member
Posts: 563
|
With ADFS_DiscOp :
Disc error 23 at :0/000C5800 at line 100
With XADFS_DiscOp :
R2=C5800 R4=1024
Resulting data is all zeroes. You forgot to change the disc address from &316 to &31B . I'm expecting disc error 18 but just want to verify it.
[Edited by sirbod at 17:33, 18/4/2012] Whilst you're at it, can you do a DiscOp 3 on track 159 as well please to confirm the ID's are invalid:
... SYS "ADFS_DiscOp", 0, 3+(z%<<6), &31B<<(z%?0), data, 0 TO a%, b%, c% PRINT "R2=";~c% OSCLI "MEMORY "+STR$~data+" +40" [Edited by sirbod at 18:13, 18/4/2012] |
|
[ Log in to reply ] |
|
Tom Walker |
Message #120120, posted by TomWalker at 17:20, 18/4/2012, in reply to message #120119 |
Member
Posts: 25
|
I'm expecting disc error 18 but just want to verify it. Nope!
With ADFS_DiscOp:
Disc error 23 at :0/000C6C00 at line 100
With XADFS_DiscOp:
R2=C6C00 R4=1024
Data is all zeroes. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120121, posted by sirbod at 17:56, 18/4/2012, in reply to message #120120 |
Member
Posts: 563
|
Disc error 23 at :0/000C6C00 Two checks further on now. This one is the nail in the coffin for the APD though:
DiscOp 1: &1400 from &A1 - 161.0 (14, 1, 2, 4, &D00C00)
That reads &1400 of a 16k sector, 1 per track, 2 heads, quad density from track 161 sector 0.
You can test this via:
DIM data 20480 FOR a%=0 TO 20479 STEP 4:data!a% = 0:NEXT DIM z% 64 FOR a%=0 TO 63 STEP 4:z%!a% = 0:NEXT z%?0=14 z%?1=1 z%?2=2 z%?3=4 z%!16=&D00C00 SYS "ADFS_DiscOp", 0, 1+(z%<<6), &A1<<(z%?0), data, &1400 TO a%, b%, c%, d%, e% PRINT "R2=";~c%;" R3=";~d%;" R4=";~e% OSCLI "MEMORY "+STR$~data+" + 1400"
If you can recreate the APD with the track in, we can see if it then works. Might be worth checking tracks 163 and 165 for quad data as well.
[Edited by sirbod at 19:03, 18/4/2012] |
|
[ Log in to reply ] |
|
Tom Walker |
Message #120123, posted by TomWalker at 18:55, 18/4/2012, in reply to message #120121 |
Member
Posts: 25
|
Disc error 23 at :0/000C6C00 Two checks further on now. This one is the nail in the coffin for the APD though:
DiscOp 1: &1400 from &A1 - 161.0 (14, 1, 2, 4, &D00C00)
That reads &1400 of a 16k sector, 1 per track, 2 heads, quad density from track 161 sector 0. That might explain why it doesn't work on a 1772. What were they thinking? They must have known this couldn't work on an A3000.
If you can recreate the APD with the track in, we can see if it then works. Might be worth checking tracks 163 and 165 for quad data as well. The standard Amiga setup can't image quad density data, at least not without one of the funny A4000 half-speed drives. Kryoflux can, but that's more involved, and will have to wait.
Incidentally, a quick look at the log file suggests track 163 also has quad density data, but 165 does not. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120125, posted by sirbod at 19:25, 18/4/2012, in reply to message #120123 |
Member
Posts: 563
|
What were they thinking? Exactly, Gordon Key probably had something to do with it going by the name of the protection: Gordian Lock
The standard Amiga setup can't image quad density data, at least not without one of the funny A4000 half-speed drives. Kryoflux can, but that's more involved, and will have to wait. Sounds like a prime candidate for the new image format . To get this far with Fire & Ice I had to code the ADFFS side of the imager, so just need a front end to actually grab it from the floppy. Anyone want to volunteer?
track 163 also has quad density data, but 165 does not. Noted, thanks for confirming.
[Edited by sirbod at 20:30, 18/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120130, posted by sirbod at 08:19, 19/4/2012, in reply to message #120123 |
Member
Posts: 563
|
Tom, a question for you, or anyone else that knows about floppy head positioning for that matter.
Gribbly's Day Out requires duplicate sector support, how did you handle the head position advancing in Arculator?
Track 159 has sectors 8 (1), 1, 2, 3, 4, 5, 6, 7, 8 (2)
It expected to read in the following order:
1. 159.4 2. 159.5 3. 159.7 4. 159.8 (1)
I can't figure out how it's stepping over 8 (2) - unless the head advances two sectors between reads. I've tried repeatedly reading in the above order, with/without time delays but it always returns 8 (1).
Currently I have the head advancing 1 sector per read with a botch for Gribbly. I did consider writing an interrupt driven head advancer, or a time based positioner - but it will need to be dead accurate for Gribbly to work 100% and does rely on Gribbly having a delay between reads.
I don't honestly see how Gribby works 100% on an actual floppy, if it fails the check it immediately reboots the machine - very nasty as it's done in the !Boot.
[Edited by sirbod at 09:20, 19/4/2012] |
|
[ Log in to reply ] |
|
Tom Walker |
Message #120132, posted by TomWalker at 08:36, 19/4/2012, in reply to message #120130 |
Member
Posts: 25
|
I think I did have to emulate disc rotation for Gribbly. Arculator advances sectors as long as the drive motor is on. |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120133, posted by sirbod at 08:37, 19/4/2012, in reply to message #120107 |
Member
Posts: 563
|
I'm down to the final APD - Sensible Soccer. Trouble is, it has me stumped. Its protection is as follows:
1. Read &200 from track 158 sector 240 in single density 2. Read &200 from track 158 sector 242 in single density 3. Read &200 from track 158 sector 240 in double density 4. Read &200 from track 158 sector 242 in double density
The problem comes with the double density reads, in both the 4th byte is incorrect and I can't for the life of me figure out why:
Read 3 should start &C000FF00 I'm seeing &C000FFE9 Read 4 should start &C451F690 I'm seeing &C451F698
As it's stored big-endian, its clearly a problem with the first byte of the sector...just need to figure out why its only affecting this APD!
EDIT: Its a bug in the MemCopy code - god knows how anything is working currently!
EDIT2: MemCopy was corrupting the 4th byte if the source was +1 from being word aligned and the destination was word aligned! It's been there since since day 1, fixed in 1.24
[Edited by sirbod at 23:03, 19/4/2012] |
|
[ Log in to reply ] |
|
Jon Abbott |
Message #120134, posted by sirbod at 08:39, 19/4/2012, in reply to message #120132 |
Member
Posts: 563
|
I think I did have to emulate disc rotation for Gribbly. Arculator advances sectors as long as the drive motor is on. Don't suppose you remember or can find out the timing for this? That will save me hours of trial and error.
Thanks
EDIT: I should add, I took a quick look though the source for 0.8 but couldn't see anything related.
PS: Are you going to fix the read track command and put out 1.0?
[Edited by sirbod at 09:48, 19/4/2012] |
|
[ Log in to reply ] |
|
Pages (8): |< <
5
> >|
|
The Icon Bar: General: ADF floppy disc image reader |
|