Shadow/Highlight

Ask anything your want about Megadrive/Genesis programming.

Moderator: BigEvilCorporation

Post Reply
Near
Very interested
Posts: 109
Joined: Thu Feb 28, 2008 4:45 pm

Shadow/Highlight

Post by Near » Sun Aug 20, 2017 2:42 pm

Trying to implement shadow/highlight. The documentation ... well, you all know.

I can't run Vectorman, because inputs aren't working in the game for some reason for me. Another bug, yay.
I can't run vdpdoc.bin, because it's known to not work on hardware, and also doesn't run for me. Expects a strange 68K bus grant behavior that breaks other games and is not correct in order to run.
So that leaves me with "Shadow-Highlight Test Program #2 (PD)" as my only possible test case, and this is what I get:

Image

First, the scrolling background on plane B on the edges of the screen are shadowed, yet in other emulators they're not.

And that ... may be the only issue? If the background were correct, the image may render correctly. But probably not.

In the docs, I noticed that Charles MacDonald got the purpose of sprite color 0x3e and 0x3f backward.
I also note that the behavior when one background plane has the priority bit set doesn't seem to actually cause the output color to become normal (not shadowed nor highlighted.)

So basically let's look at that scrolling background layer around the edges of the screen ...

Plane B contains the scrolling layer. Priority is always zero, and the color of the tiles comes out as 1 or 2 for the pattern drawn.
Plane A always has priority zero, and color zero in this area.
Sprite always has priority zero, and color zero in this area.

Per Charles MacDonald:

"Background tiles are shown at half intensity, or at normal intensity if their priority bit is set."

Per md.squee.co:

"An entry in a name table is by default shadowed with the priority bit cleared."

According to both of these, the scrolling plane B layer around the edges of the screen in this test ROM should always be shadowed, as I am rendering.
Yet in Mednafen, BlastEm, etc ... it is not.

So what am I missing?

My code, along with where I pulled notes from. This codepath is only hit when mode register 4 shadow/highlight is enabled.

Code: Select all

  auto output = io.backgroundColor;
//The backdrop is shown at half intensity, while the overscan region outside
//of the display area is always shown at normal intensity.
  enum : uint { Normal, Shadow, Highlight } mode = Shadow;

  if(!planeB.output.priority) if(auto color = planeB.output.color) {
    output = color;
//Background tiles are shown at half intensity, or at normal intensity if their priority bit is set.
    mode = Shadow;
  }
  if(!planeA.output.priority) if(auto color = planeA.output.color) {
    output = color;
    mode = Shadow;
  }
  if(!sprite.output.priority) if(auto color = sprite.output.color) {
//Sprites with colour 15 of palette 4 will highlight the underlying pixels -
//if the underlying pixel is shadowed, it will be displayed normally.
//If it's already normal, it will be highlighted.
//** note this contradicts Charles MacDonald:
//Pixels with color 3Eh are not drawn. Instead, the underlying pixel
//(from the backdrop or background) will be shown at half intensity.
         if(color == 0x3e) mode = mode == Shadow ? Normal : Highlight;
//Sprites with colour 16 of palette 4 will shadow the underlying pixels -
//if the underlying pixel is normal, it will be shadowed. If it is shadowed, nothing will occur.
//If it's highlighted, it will be displayed normally.
//** same
//Pixels with color 3Fh are not drawn. Instead, the underlying pixel
//(from the backdrop or background) will be shown at double intensity.
    else if(color == 0x3f) mode = Shadow;
    else {
      output = color;
//Colors 0Eh, 1Eh, 2Eh, are always shown at normal intensity regardless of priority.
//I'd say this a bug rather than a feature.
      mode = color == 0x0e || color == 0x1e || color == 0x2e ? Normal : Shadow;
    }
  }
  if( planeB.output.priority) if(auto color = planeB.output.color) {
    output = color;
//Background tiles are shown at half intensity, or at normal intensity if their priority bit is set.
    mode = Normal;
  }
  if( planeA.output.priority) if(auto color = planeA.output.color) {
    output = color;
    mode = Normal;
  }
  if( sprite.output.priority) if(auto color = sprite.output.color) {
         if(color == 0x3e) mode = mode == Shadow ? Normal : Highlight;
    else if(color == 0x3f) mode = Shadow;
    else {
      output = color;
      mode = Normal;
    }
  }

  auto color = cram.read(output);

//Background tiles are shown at half intensity, or at normal intensity if their priority bit is set.
//In the latter case, this affects all other graphics elements in the region (usually 8x8) that the
//tile takes up, regardless of the tile containing opaque pixels or not.
//*** this is flawed. If we do this, shadow mode breaks on the fourth column of colors.
//if(planeB.output.priority || planeA.output.priority) mode = Normal;

  //this is lazy, I know I'm cutting 50% of the color precision away, and that the actual colors are non-linear
  //don't worry, when I get the demo working I'll make a proper LUT for all three modes
  if(mode == Shadow   ) color = (color >> 1) & 0b011011011;
  if(mode == Highlight) color = (color << 1) & 0b110110110;

  outputPixel(color);
}

Miquel
Very interested
Posts: 514
Joined: Sat Jul 30, 2016 12:33 am

Re: Shadow/Highlight

Post by Miquel » Sun Aug 20, 2017 4:07 pm

Are you asking who is right? Test on real hardware, that's the irrefutable truth.

I think what you are asking is how is border colour at the borders affected by Highlight/Shadow, is it?

In my game on real hardware, all plane A is in high priority and the border colour is also in Highlight.

As soon I have the means I can try other configurations...
HELP. Spanish TVs are brain washing people to be hostile to me.

Sik
Very interested
Posts: 939
Joined: Thu Apr 10, 2008 3:03 pm
Contact:

Re: Shadow/Highlight

Post by Sik » Sun Aug 20, 2017 5:36 pm

The Charles McDonald document is seriously wrong when it comes to describing S/H (you can tell it's a really old doc). Especially when shadow and highlight interact (which happens when scrolls are shadowed and a highlight sprite goes on top).

The basic rules are (per pixel), after you calculated the color index for all three layers (meaning that sprites have been collapsed into a single layer already):
  • If any layer is high priority, default to normal, otherwise default to shadowed.
  • If visible layer is sprite palette 3 color 14, make sprite transparent and brighten the pixel.
  • If visible layer is sprite palette 3 color 15, make sprite transparent and darken the pixel.
Regarding the last two: a pixel can assume three states (highlight, normal, shadowed), so when I mean brighten/darken, I just mean moving up/down one state (also just to make clear the edge case: shadow + shadow = shadow).

If you just want a list of all possible values:
  • normal + normal = normal
  • normal + highlight = highlight
  • normal + shadow = shadow
  • shadow + normal = shadow
  • shadow + highlight = normal
  • shadow + shadow = shadow
There's an issue regarding color 14 outside sprite highlight (which makes it show up normal even if it should default to shadowed). I don't recall the exact details, every time I try to bring it up I get it wrong =P But it's the only way to have a bright color in an otherwise shadowed graphic, which probably has its interesting uses.

By the way, don't forget about night mode in Sonic 2 (enable debug mode and hold down C while entering a level from the level select).
Sik is pronounced as "seek", not as "sick".

Near
Very interested
Posts: 109
Joined: Thu Feb 28, 2008 4:45 pm

Re: Shadow/Highlight

Post by Near » Sun Aug 20, 2017 7:31 pm

Much appreciated! These notes and help from Cydrak led to a working, minimal implementation :D

The main thing that was breaking the demo originally was that I wasn't updating the background priority bit when the color was transparent. That's not something that's normally done on other systems I've emulated. After that, the docs had the "if either plane A or plane B priority is set, everything becomes normal" part wrong ... that's not actually true. If sprite has priority set, that doesn't happen.

Then Cydrak cut the code size in half with really clever knowledge of boolean logic reduction. Seems to work great now :D

Code: Select all

auto VDP::run() -> void {
  if(!io.displayEnable) return outputPixel(0);
  if(state.vcounter >= screenHeight()) return outputPixel(0);

  auto& planeA = window.isWindowed(state.hdot, state.vcounter) ? window : this->planeA;
  planeA.run(state.hdot, state.vcounter);
  planeB.run(state.hdot, state.vcounter);
  sprite.run(state.hdot, state.vcounter);

  Pixel z = {io.backgroundColor, 0};
  Pixel a = planeA.output;
  Pixel b = planeB.output;
  Pixel s = sprite.output;

  auto& bg = a.above() || a.color && !b.above() ? a : b.color ? b : z;
  auto& fg = s.above() || s.color && !a.above() && !b.above() ? s : bg;
  uint mode = a.priority || b.priority;

  if(&fg == &s) switch(s.color) {
  case 0x0e:
  case 0x1e:
  case 0x2e: mode  = 1; break;
  case 0x3e: mode += 1; fg = bg; break;
  case 0x3f: mode  = 0; fg = bg; break;
  default:   mode |= s.priority; break;
  }

  auto color = cram.read(fg.color);
  if(!io.shadowHighlightEnable) mode = 1;
  if(mode == 0) color = (color >> 1) & 0b011011011;  //todo: separate LUTs for more precision
  if(mode == 2) color = (color << 1) & 0b110110110;  //and also for non-linear color response
  outputPixel(color);

  state.hdot++;
}

Sik
Very interested
Posts: 939
Joined: Thu Apr 10, 2008 3:03 pm
Contact:

Re: Shadow/Highlight

Post by Sik » Mon Aug 21, 2017 1:01 am

byuu wrote:
Sun Aug 20, 2017 7:31 pm
The main thing that was breaking the demo originally was that I wasn't updating the background priority bit when the color was transparent.
Oh, um, I should probably get around explaining that the next time (・-・;) (the VDP always processes all layers including transparent pixels, the only time they take effect is when deciding whose color index to display)

EDIT: oh, and nitpick but:
byuu wrote:
Sun Aug 20, 2017 7:31 pm

Code: Select all

  Pixel z = {io.backgroundColor, 0};
We normally call that layer G =P (that comes from Sega's docs)
Sik is pronounced as "seek", not as "sick".

flamewing
Very interested
Posts: 56
Joined: Tue Sep 23, 2014 2:39 pm
Location: France

Re: Shadow/Highlight

Post by flamewing » Mon Aug 21, 2017 7:48 am

Sik wrote:
Sun Aug 20, 2017 5:36 pm
There's an issue regarding color 14 outside sprite highlight (which makes it show up normal even if it should default to shadowed). I don't recall the exact details, every time I try to bring it up I get it wrong =P But it's the only way to have a bright color in an otherwise shadowed graphic, which probably has its interesting uses.
That is pretty much it: a sprite pixel using color 14 of any palette line always shows up as normal, and is never shadowed. If you have a black color in your sprite palette lines, you should put it on color 14 because the sprite will then show up correctly with shadow/highlight active.

Sik
Very interested
Posts: 939
Joined: Thu Apr 10, 2008 3:03 pm
Contact:

Re: Shadow/Highlight

Post by Sik » Mon Aug 21, 2017 12:35 pm

Or use it to store the color for a light (e.g. Pulseman's watch) so it shows up lit while in shadow =P (ironic example since Pulseman doesn't ever use S/H)

What about when color 14 shows up in the tilemaps though?
Sik is pronounced as "seek", not as "sick".

flamewing
Very interested
Posts: 56
Joined: Tue Sep 23, 2014 2:39 pm
Location: France

Re: Shadow/Highlight

Post by flamewing » Tue Aug 22, 2017 11:09 am

Sik wrote:
Mon Aug 21, 2017 12:35 pm
Or use it to store the color for a light (e.g. Pulseman's watch) so it shows up lit while in shadow =P (ironic example since Pulseman doesn't ever use S/H)
That is a very neat idea.
Sik wrote:
Mon Aug 21, 2017 12:35 pm
What about when color 14 shows up in the tilemaps though?
If memory serves, tilemaps (and the background) show up correctly, it is only sprites which get the weird behavior.

TmEE co.(TM)
Very interested
Posts: 2440
Joined: Tue Dec 05, 2006 1:37 pm
Location: Estonia, Rapla City
Contact:

Re: Shadow/Highlight

Post by TmEE co.(TM) » Tue Aug 22, 2017 11:38 am

Yes, it only affects sprites, BGs work normally.
Mida sa loed ? Nagunii aru ei saa ;)
http://www.tmeeco.eu
Files of all broken links and images of mine are found here : http://www.tmeeco.eu/FileDen

Sik
Very interested
Posts: 939
Joined: Thu Apr 10, 2008 3:03 pm
Contact:

Re: Shadow/Highlight

Post by Sik » Tue Aug 22, 2017 11:39 am

OK that probably explains why I kept getting it wrong all the time, the behavior is different depending on the layer =P (I guess it makes sense since color 14 only has special meaning in sprites and not tilemaps)
Sik is pronounced as "seek", not as "sick".

r57shell
Very interested
Posts: 478
Joined: Sun Dec 23, 2012 1:30 pm
Location: Russia
Contact:

Re: Shadow/Highlight

Post by r57shell » Tue Aug 22, 2017 8:37 pm

Just in case if you need some tests.
viewtopic.php?f=8&t=1585#p21261
Here is ROM which has few tests of shadow highlight.
One is where it asks for "full rect is same color"
and other ...
Anyway, just play with it.

Maybe this is how kabuto inspired his vertical counter overflow trick
viewtopic.php?f=22&t=2604
But maybe not. Only kabuto knows. :D
Image

Miquel
Very interested
Posts: 514
Joined: Sat Jul 30, 2016 12:33 am

Re: Shadow/Highlight

Post by Miquel » Wed Aug 30, 2017 5:15 pm

Sik wrote:
Mon Aug 21, 2017 12:35 pm
Or use it to store the color for a light (e.g. Pulseman's watch) so it shows up lit while in shadow =P (ironic example since Pulseman doesn't ever use S/H)
So that's it! That's what happens in the game I'm developing.

The color 14 is for white, used for the eyes, and whenever the main character goes into dark his eyes pop up. Creates kind of nice effect, but it wasn't done on propose... :D it's funny...

If I remember correctly I did it that way because if I change palette to the last one, white color converts itself to highlight, and black color (15) changes to dark. So, when drawing sprites makes sense.

It's really funny... for me at last...
HELP. Spanish TVs are brain washing people to be hostile to me.

Chilly Willy
Very interested
Posts: 2984
Joined: Fri Aug 17, 2007 9:33 pm

Re: Shadow/Highlight

Post by Chilly Willy » Thu Aug 31, 2017 12:11 am

Eyeballs floating through the dark... that would make a cool effect for some games. :D

Miquel
Very interested
Posts: 514
Joined: Sat Jul 30, 2016 12:33 am

Re: Shadow/Highlight

Post by Miquel » Sat Sep 02, 2017 7:43 pm

Nothing spectacular, but I think is noticeable:
Attachments
InShadow.jpg
InShadow.jpg (145.26 KiB) Viewed 13197 times
OutShadow.jpg
OutShadow.jpg (170.08 KiB) Viewed 13197 times
HELP. Spanish TVs are brain washing people to be hostile to me.

Post Reply