Page 1 of 1

Shadow/Highlight

Posted: Sun Aug 20, 2017 2:42 pm
by Near
Trying to implement shadow/highlight. The documentation ... well, you all know.

I can't run Vectorman, because inputs aren't working in the game for some reason for me. Another bug, yay.
I can't run vdpdoc.bin, because it's known to not work on hardware, and also doesn't run for me. Expects a strange 68K bus grant behavior that breaks other games and is not correct in order to run.
So that leaves me with "Shadow-Highlight Test Program #2 (PD)" as my only possible test case, and this is what I get:

Image

First, the scrolling background on plane B on the edges of the screen are shadowed, yet in other emulators they're not.

And that ... may be the only issue? If the background were correct, the image may render correctly. But probably not.

In the docs, I noticed that Charles MacDonald got the purpose of sprite color 0x3e and 0x3f backward.
I also note that the behavior when one background plane has the priority bit set doesn't seem to actually cause the output color to become normal (not shadowed nor highlighted.)

So basically let's look at that scrolling background layer around the edges of the screen ...

Plane B contains the scrolling layer. Priority is always zero, and the color of the tiles comes out as 1 or 2 for the pattern drawn.
Plane A always has priority zero, and color zero in this area.
Sprite always has priority zero, and color zero in this area.

Per Charles MacDonald:

"Background tiles are shown at half intensity, or at normal intensity if their priority bit is set."

Per md.squee.co:

"An entry in a name table is by default shadowed with the priority bit cleared."

According to both of these, the scrolling plane B layer around the edges of the screen in this test ROM should always be shadowed, as I am rendering.
Yet in Mednafen, BlastEm, etc ... it is not.

So what am I missing?

My code, along with where I pulled notes from. This codepath is only hit when mode register 4 shadow/highlight is enabled.

Code: Select all

  auto output = io.backgroundColor;
//The backdrop is shown at half intensity, while the overscan region outside
//of the display area is always shown at normal intensity.
  enum : uint { Normal, Shadow, Highlight } mode = Shadow;

  if(!planeB.output.priority) if(auto color = planeB.output.color) {
    output = color;
//Background tiles are shown at half intensity, or at normal intensity if their priority bit is set.
    mode = Shadow;
  }
  if(!planeA.output.priority) if(auto color = planeA.output.color) {
    output = color;
    mode = Shadow;
  }
  if(!sprite.output.priority) if(auto color = sprite.output.color) {
//Sprites with colour 15 of palette 4 will highlight the underlying pixels -
//if the underlying pixel is shadowed, it will be displayed normally.
//If it's already normal, it will be highlighted.
//** note this contradicts Charles MacDonald:
//Pixels with color 3Eh are not drawn. Instead, the underlying pixel
//(from the backdrop or background) will be shown at half intensity.
         if(color == 0x3e) mode = mode == Shadow ? Normal : Highlight;
//Sprites with colour 16 of palette 4 will shadow the underlying pixels -
//if the underlying pixel is normal, it will be shadowed. If it is shadowed, nothing will occur.
//If it's highlighted, it will be displayed normally.
//** same
//Pixels with color 3Fh are not drawn. Instead, the underlying pixel
//(from the backdrop or background) will be shown at double intensity.
    else if(color == 0x3f) mode = Shadow;
    else {
      output = color;
//Colors 0Eh, 1Eh, 2Eh, are always shown at normal intensity regardless of priority.
//I'd say this a bug rather than a feature.
      mode = color == 0x0e || color == 0x1e || color == 0x2e ? Normal : Shadow;
    }
  }
  if( planeB.output.priority) if(auto color = planeB.output.color) {
    output = color;
//Background tiles are shown at half intensity, or at normal intensity if their priority bit is set.
    mode = Normal;
  }
  if( planeA.output.priority) if(auto color = planeA.output.color) {
    output = color;
    mode = Normal;
  }
  if( sprite.output.priority) if(auto color = sprite.output.color) {
         if(color == 0x3e) mode = mode == Shadow ? Normal : Highlight;
    else if(color == 0x3f) mode = Shadow;
    else {
      output = color;
      mode = Normal;
    }
  }

  auto color = cram.read(output);

//Background tiles are shown at half intensity, or at normal intensity if their priority bit is set.
//In the latter case, this affects all other graphics elements in the region (usually 8x8) that the
//tile takes up, regardless of the tile containing opaque pixels or not.
//*** this is flawed. If we do this, shadow mode breaks on the fourth column of colors.
//if(planeB.output.priority || planeA.output.priority) mode = Normal;

  //this is lazy, I know I'm cutting 50% of the color precision away, and that the actual colors are non-linear
  //don't worry, when I get the demo working I'll make a proper LUT for all three modes
  if(mode == Shadow   ) color = (color >> 1) & 0b011011011;
  if(mode == Highlight) color = (color << 1) & 0b110110110;

  outputPixel(color);
}

Re: Shadow/Highlight

Posted: Sun Aug 20, 2017 4:07 pm
by Miquel
Are you asking who is right? Test on real hardware, that's the irrefutable truth.

I think what you are asking is how is border colour at the borders affected by Highlight/Shadow, is it?

In my game on real hardware, all plane A is in high priority and the border colour is also in Highlight.

As soon I have the means I can try other configurations...

Re: Shadow/Highlight

Posted: Sun Aug 20, 2017 5:36 pm
by Sik
The Charles McDonald document is seriously wrong when it comes to describing S/H (you can tell it's a really old doc). Especially when shadow and highlight interact (which happens when scrolls are shadowed and a highlight sprite goes on top).

The basic rules are (per pixel), after you calculated the color index for all three layers (meaning that sprites have been collapsed into a single layer already):
  • If any layer is high priority, default to normal, otherwise default to shadowed.
  • If visible layer is sprite palette 3 color 14, make sprite transparent and brighten the pixel.
  • If visible layer is sprite palette 3 color 15, make sprite transparent and darken the pixel.
Regarding the last two: a pixel can assume three states (highlight, normal, shadowed), so when I mean brighten/darken, I just mean moving up/down one state (also just to make clear the edge case: shadow + shadow = shadow).

If you just want a list of all possible values:
  • normal + normal = normal
  • normal + highlight = highlight
  • normal + shadow = shadow
  • shadow + normal = shadow
  • shadow + highlight = normal
  • shadow + shadow = shadow
There's an issue regarding color 14 outside sprite highlight (which makes it show up normal even if it should default to shadowed). I don't recall the exact details, every time I try to bring it up I get it wrong =P But it's the only way to have a bright color in an otherwise shadowed graphic, which probably has its interesting uses.

By the way, don't forget about night mode in Sonic 2 (enable debug mode and hold down C while entering a level from the level select).

Re: Shadow/Highlight

Posted: Sun Aug 20, 2017 7:31 pm
by Near
Much appreciated! These notes and help from Cydrak led to a working, minimal implementation :D

The main thing that was breaking the demo originally was that I wasn't updating the background priority bit when the color was transparent. That's not something that's normally done on other systems I've emulated. After that, the docs had the "if either plane A or plane B priority is set, everything becomes normal" part wrong ... that's not actually true. If sprite has priority set, that doesn't happen.

Then Cydrak cut the code size in half with really clever knowledge of boolean logic reduction. Seems to work great now :D

Code: Select all

auto VDP::run() -> void {
  if(!io.displayEnable) return outputPixel(0);
  if(state.vcounter >= screenHeight()) return outputPixel(0);

  auto& planeA = window.isWindowed(state.hdot, state.vcounter) ? window : this->planeA;
  planeA.run(state.hdot, state.vcounter);
  planeB.run(state.hdot, state.vcounter);
  sprite.run(state.hdot, state.vcounter);

  Pixel z = {io.backgroundColor, 0};
  Pixel a = planeA.output;
  Pixel b = planeB.output;
  Pixel s = sprite.output;

  auto& bg = a.above() || a.color && !b.above() ? a : b.color ? b : z;
  auto& fg = s.above() || s.color && !a.above() && !b.above() ? s : bg;
  uint mode = a.priority || b.priority;

  if(&fg == &s) switch(s.color) {
  case 0x0e:
  case 0x1e:
  case 0x2e: mode  = 1; break;
  case 0x3e: mode += 1; fg = bg; break;
  case 0x3f: mode  = 0; fg = bg; break;
  default:   mode |= s.priority; break;
  }

  auto color = cram.read(fg.color);
  if(!io.shadowHighlightEnable) mode = 1;
  if(mode == 0) color = (color >> 1) & 0b011011011;  //todo: separate LUTs for more precision
  if(mode == 2) color = (color << 1) & 0b110110110;  //and also for non-linear color response
  outputPixel(color);

  state.hdot++;
}

Re: Shadow/Highlight

Posted: Mon Aug 21, 2017 1:01 am
by Sik
byuu wrote:
Sun Aug 20, 2017 7:31 pm
The main thing that was breaking the demo originally was that I wasn't updating the background priority bit when the color was transparent.
Oh, um, I should probably get around explaining that the next time (・-・;) (the VDP always processes all layers including transparent pixels, the only time they take effect is when deciding whose color index to display)

EDIT: oh, and nitpick but:
byuu wrote:
Sun Aug 20, 2017 7:31 pm

Code: Select all

  Pixel z = {io.backgroundColor, 0};
We normally call that layer G =P (that comes from Sega's docs)

Re: Shadow/Highlight

Posted: Mon Aug 21, 2017 7:48 am
by flamewing
Sik wrote:
Sun Aug 20, 2017 5:36 pm
There's an issue regarding color 14 outside sprite highlight (which makes it show up normal even if it should default to shadowed). I don't recall the exact details, every time I try to bring it up I get it wrong =P But it's the only way to have a bright color in an otherwise shadowed graphic, which probably has its interesting uses.
That is pretty much it: a sprite pixel using color 14 of any palette line always shows up as normal, and is never shadowed. If you have a black color in your sprite palette lines, you should put it on color 14 because the sprite will then show up correctly with shadow/highlight active.

Re: Shadow/Highlight

Posted: Mon Aug 21, 2017 12:35 pm
by Sik
Or use it to store the color for a light (e.g. Pulseman's watch) so it shows up lit while in shadow =P (ironic example since Pulseman doesn't ever use S/H)

What about when color 14 shows up in the tilemaps though?

Re: Shadow/Highlight

Posted: Tue Aug 22, 2017 11:09 am
by flamewing
Sik wrote:
Mon Aug 21, 2017 12:35 pm
Or use it to store the color for a light (e.g. Pulseman's watch) so it shows up lit while in shadow =P (ironic example since Pulseman doesn't ever use S/H)
That is a very neat idea.
Sik wrote:
Mon Aug 21, 2017 12:35 pm
What about when color 14 shows up in the tilemaps though?
If memory serves, tilemaps (and the background) show up correctly, it is only sprites which get the weird behavior.

Re: Shadow/Highlight

Posted: Tue Aug 22, 2017 11:38 am
by TmEE co.(TM)
Yes, it only affects sprites, BGs work normally.

Re: Shadow/Highlight

Posted: Tue Aug 22, 2017 11:39 am
by Sik
OK that probably explains why I kept getting it wrong all the time, the behavior is different depending on the layer =P (I guess it makes sense since color 14 only has special meaning in sprites and not tilemaps)

Re: Shadow/Highlight

Posted: Tue Aug 22, 2017 8:37 pm
by r57shell
Just in case if you need some tests.
viewtopic.php?f=8&t=1585#p21261
Here is ROM which has few tests of shadow highlight.
One is where it asks for "full rect is same color"
and other ...
Anyway, just play with it.

Maybe this is how kabuto inspired his vertical counter overflow trick
viewtopic.php?f=22&t=2604
But maybe not. Only kabuto knows. :D

Re: Shadow/Highlight

Posted: Wed Aug 30, 2017 5:15 pm
by Miquel
Sik wrote:
Mon Aug 21, 2017 12:35 pm
Or use it to store the color for a light (e.g. Pulseman's watch) so it shows up lit while in shadow =P (ironic example since Pulseman doesn't ever use S/H)
So that's it! That's what happens in the game I'm developing.

The color 14 is for white, used for the eyes, and whenever the main character goes into dark his eyes pop up. Creates kind of nice effect, but it wasn't done on propose... :D it's funny...

If I remember correctly I did it that way because if I change palette to the last one, white color converts itself to highlight, and black color (15) changes to dark. So, when drawing sprites makes sense.

It's really funny... for me at last...

Re: Shadow/Highlight

Posted: Thu Aug 31, 2017 12:11 am
by Chilly Willy
Eyeballs floating through the dark... that would make a cool effect for some games. :D

Re: Shadow/Highlight

Posted: Sat Sep 02, 2017 7:43 pm
by Miquel
Nothing spectacular, but I think is noticeable: