Timings issue when code runs in external RAM

[sirdavid2]

While writing a demo for SAM I have noticed a difference between MIST core and SimCoupe/real hardware when the code is executed from the external RAM. I have prepared the small example code to show it:
https://www.dropbox.com/s/bh4nj29tdvtcp4z/overscan.zip?dl=0

Screen looks significantly different than it should.
When it runs in the internal RAM, there is no difference between MIST core and others.
Maybe the same problem causes an issue in NoAGA demo.

[sorgelig]

It's hard to remember all implementation details as i did it several years ago :) If i remember correctly i've implemented extended RAM as a simple RAM extension without specific timings as there were no tests for that.
Unlike the real HW where extram is separate device, on MiST/MiSTer it's the same RAM chip, so it may be limited by video access sharing and basically works with the same timings as internal SAM Couple RAM.

I don't have real SAM Coupe so i can't test it. Describe the difference, so i will see if it possible to fix.

[sirdavid2]

It's not easy to describe, but I'll try. The code in external RAM in MIST core runs faster than in internal, as it should be. I'm going to write another test to check if it is exactly as fast as it should or not. The difference I see for now is the exact moment of the color change after out to the CLUT register. This causes a shift of the color change on screen.
You can test it on SimCoupe emulator (v 1.1), it shows everything correctly.

[sorgelig]

would be good to get a good description of extram timings.

[sirdavid2]

It seems that external RAM runs in exactly proper speed overall, but something somehow has to cause that shift... I have made some more tests and the screen effect while the code is running in internal RAM always looks exactly the same on both MIST and SimCoupe, but in external RAM is not. Sometimes it is the same, but more often it's not. I have to do more tests...

Could you explain why the screen as a whole is shifted to the left by... 8 points? Or less? Border should be the same width on the letf and right but it's not. Maybe it has something to do with it?

[sorgelig]

Actually i don't use MiST already long time. I'm using MiSTer with HDMI output where border is perfect. I remember on MiST it was ok on my old setup.
I've run your test on SimCoupe. I see some pattern there but i would say it's quite weird test as it's hard to see any shift of such pattern. It has so many chaotic colors. Visually it's hard to detect the nature of problem.
Since core uses DRAM, it needs to refresh it time after time - may be this cause the problem. Or may be video access interfering the CPU access.. So, it's hard to say. I need some more representative test. May be some vertical single color stripes so at least can see if core is faster or slower and what is the pattern of slower/faster cycles.

[sirdavid2]

For now it turned out that I don't have to use external RAM to spot the difference. The same is when I use ROM. When all code runs from internal RAM, the final effect is the same on every platform. But when instead of out (c),a I use call to ROM, to: out (c),a , ret , MIST core starts to differ from the real SAM or SimCoupe emulator.

[sorgelig]

Well, for cycle accuracy, fix would be good. From other side, you don't have to execute code from extram. SAM Coupe gives enough sync methods like raster interrupt, to not depend on extram timings.

[sirdavid2]

It's quite the opposite. I have to execute code from external RAM to get better (faster, smoother) effects. It's just like fast RAM on Amiga. Accurate timings are necessary for effects based on changing colour definitions on the fly.
I'll try to prepare the sample screen where the difference will be clearly visible. I wonder how you managed to get proper timings for internal RAM. Could you tell what part of the core code is responsible for internal RAM timings (wchich file and lines)? Maybe I could try to fix it myself if I knew that.

[sorgelig]

Extram on real HW is physically separate device which is not interfering by anything, and only CPU have access to it. In the core, it's the same device as internal RAM. Core doesn't limit speed while RAM is accessed as extram, so it works at max speed, but it's slowed down by video memory access for rendering. While you can slow down the extram, you can't make it work faster by one line. It may need serious changes. Extram probably uses DRAM, so it also needs some refresh cycles.

[sorgelig]

timings for internal RAM are controlled by contention model:

SAMCoupe_MIST/SamCoupe.sv

Line 172 in c8e7a77

// Contention model

Of course it's correct for internal ram since it's well tested.

[sirdavid2]

The reason is rather ASIC contention, not memory. Writing to ASIC port can last longer because of the contention. I think a slight difference when ASIC should be busy can result in that shift in the moment of colour change. These moments of colour changes are correct most of the time, but one in several changes is too early or too late (I haven't figured out the direction yet). Does my way of thinking make sense?

[sorgelig]

ASIC is accessed not only when code is in extram.

[sirdavid2]

I know that and that's why I think it could be ASIC. But meanwhile I have discovered a bug in z80 code in SAM core - the command CPIR takes the same time as LDIR, but should be faster (I use CPIR or LDIR for delay loops). CPI is ok.

[sorgelig]

According to Z80 user manual both instructions use the same amount of clocks.
Also T80 passes ZXAll test which includes timings for all instructions.

[sirdavid2]

Right! I should have checked it first! It was SimCoupe (and my) fault as the real SAM behaves just like MIST core in my cpir/ldir test.

[sirdavid2]

Recently I've been busy busy trying to catch a bug in ZX Spectrum core, but finally I have prepared two samples where the difference between MIST core and other platforms is clearly visible.
https://www.dropbox.com/s/kavujpjhzq4ju7v/Overscan2.zip?dl=0
First sample shows some vertical bars on screen, two of which are of different width than they should be. Red one shoud be 2 chars wide (and is 1 char wide on MIST core) and white one next to red should be 2, but is 3. Overall the speed seems to be correct, but the color changes sometimes don't hit the proper moment.
The second sample shows the bars which on MIST are all shifted by 1 char to the right. All the color changes are a bit late but the first one. There is one top line which is longer by 1 char, so the 1st change is in the right moment, but all subsequent changes are delayed by 1 char.

[sorgelig]

looking at regular pattern in first test it seems the problem because of shared memory.
Did you test it on real SAM Coupe with expanded memory? I compare it with simcoupe and not sure it it matches the real HW.

[sirdavid2]

Yes, I have real SAM with external RAM and SimCoupe exactly matches the HW.

[sorgelig]

btw, while we talking about MiST here, i'm actually don't develop for MiST.
The same core (with may be few differences) i use on MiSTer. I don't have MiST development environment now.

[sirdavid2]

So how my sample code work with MiSTer? Correctly or not?

[sorgelig]

also wrong, same as MiST as code is basically the same.

[sorgelig]

In your test which pages of extram you use? I want to try BRAM for those pages to see if it will fix the issue.

[sirdavid2]

I use pages 0 and 1

[sorgelig]

can you give me source code of first test from overscan2.zip?
Actually i need only that loop outputing the color lines.

[sirdavid2]

rept 300
out (c),a
out (c),e
inc hl
out (c),a
inc hl
out (c),e
nop
out (c),d
endm

Where inc hl and nop are only to make wider bars and synchronization. Inc hl is here only as a command that lasts 6 clock cycles (a bit more than nop).

[sorgelig]

i've got some clue. it's about contention slots.. Strange it works fine when extram is not used.
But one of mnedemo is broken after fix. I need to find a working config.

[sorgelig]

I now getting almost correct look of your test, but it's shifted one char to the right.
Can you confirm on real HW you get the same positions as on simcoupe?

[sirdavid2]

Yes. I have compared them all before sharing. And I don't remember to see any difference in graphics between simcoupe and real HW.

[sorgelig]

it makes me think there is another stage in redering pipeline which is not implemented yet.
Probably the paper area is rendered with 8 dots (1 char) delay additionally to 4 dots delay (which is implemented).
Is there a demos where paper area is strictly matching the border area horizontally?

[sirdavid2]

It's easier for me to write an example:
https://www.dropbox.com/s/k3vvc3isaoe4g38/Overscan3.zip?dl=0
Can it be like that?

[sorgelig]

SamCoupe.zip
Try this build. I've tested same changes on MiSTer - your tests work ok. Bit I didn't test MiST.
If it will work fine, then test other demos you think sensitive to timings and using some border effects.

[sirdavid2]

Great! All my tests and demos look ok now. Thanks! You can update the core code. The only "but" is that it unexpectedly haven't helped in NoAGA demo. Maybe there is another problem. I'll try to debug this demo and prepare a sample with still image showing what's wrong.

[sirdavid2]

So I have made a sample code with still frame from NoAGA demo, where the issue is clearly visible:
https://www.dropbox.com/s/6gdm2323thylmys/NoAGA-bug.zip?dl=0
There is a dotted line on the right that shouldn't be here. Why dotted? The screen is covered by vertical bars, every second bar is dotted. You can see screen memory content (also magnified) on attached screenshot from my SCADM utility. The line is thin because the bars do not match 8-pixel chars, which is also visible on my screenshot. The final effect is created by changing palette every screen line by repeated OUTI, as seen on "debug" screenshots. You can see where the raster is (brighter dot near the bottom-right corner) after the last OUTI and after the next command - right after the buggy line. Unlike in my samples, this time all the colors are changing, but it doesn't have to matter. Now I have to modify the screen and see if I can maka a wider bugged line and how wide can it be.
Edit: NoAGA uses external RAM, but I have to check how the same situation will look like with internal RAM.

[sorgelig]

while i understand purpose of screenshots, i don't understand what app on disk supposed to show.
I can see screens on PC.

[sirdavid2]

App on disk can be run on MiST/MiSTer and compared with simcoupe (or real HW if one has it). It also allows to check how the core modifications affect the result on screen. I can't do a clear screenshot from MiST, so it is better to run such test disk and see how the bug looks like.

[sorgelig]

does it mean, it simulates the same timings with colors? So i can tweak the timings and if it will be correct, then column of dots will disappear?

[sirdavid2]

I don't know what you mean by "the same timings". The same as what? As my earlier tests? I don't know how to compare my previous tests with this one. I'm yet to check how wide that column can be and where it comes from. And yes, if the timigs were correct, then column of dots should disappear.

[sorgelig]

And yes, if the timigs were correct, then column of dots should disappear

Doesn't seem correct. In simcoupe your NoAGA-bug.dsk also has column of dots.

[sirdavid2]

No, it hasn't. But... I have checked all simcoupe versions I have and 1.0 is ok, the newest compilaton of 1.1 is ok, but every compilations between them show that line! I had no idea about it! I don't know how to explain that, because NoAGA demo works ok on every compilation! But I have to add that there is no line on real HW, I have just checked it.

[sorgelig]

ok, fresh build of simcoupe has no that line.
I don't know if i will have enough motivation to look into that. I have no real HW, so i don't have much info to fix it.

[sorgelig]

SamCoupe.zip
try this build

[sirdavid2]

Yes! Finally NoAGA works as it should. As everything else. Thank you!