It's finally time to talk about the software for this thing, and all of the woe that has come along with that.
A Quick Refresher
As previously noted, despite the Egg Boy Color being a legitimate GBC with the OG GBC chips, there is software involved, which handles three key things(and one future thing)
1. Converting the image signal from the GBC CPU into a compatible signal for the new display
2. LCD brightness control
3. Volume control
4. In the next revision, control over the GBC CPU's reset signal
This update will cover the first three things, but mostly the first.
Driving the LCD
In principle -- and largely in practice -- pulling the GBC image data and driving the LCD is a relatively simple affair.
The Game Boy Color uses a simple parallel RGB interface, which boils down to a VSYNC, HSYNC, pixel clock, and 15-bits of color data that update prior to each pixel clock pulse. In my case, I use a couple of PIO programs on the RP2040 to monitor the timing signals, pull in color data, and then dump that into the TX FIFO queue.
External to the PIO, there are two chained DMA routines that watch the TX FIFO and move the incoming data into a frame buffer until the buffer size has been hit. The reason for the chained DMAs is because I'm using a double buffered setup -- each DMA is assigned to one of the buffers, and during the DMA completion handler, the active buffer flips and the next DMA is kicked off.
I didn't feel like faffing about with writing my own software for interfacing with the LCD, so I opted to use the TFT_eSPI library, which supports SPI communication with a wide array of LCD driver ICs, and has a bunch of helpful features baked in, like window addressing for writes. As soon as a frame is ready -- and the TE signal from the LCD indicates the start of a new frame -- the contents of the most recently completed buffer are sent to the LCD.
The Scaling Problem
As great as it would be, modern LCDs with a resolution of 160 x 144 are essentially non-existent, which requires the use of something with a higher resolution. As previously noted, I chose a small 320 x 320 panel that allows me to use a clean 2x integer scale to fill the entire screen.
Unfortunately, this is where things start becoming difficult. Integer scaling is by itself a simple thing -- taking an image of a given size and scaling it into an appropriately sized buffer is easy. However, doing so within the RAM and CPU limitations of an MCU becomes more difficult.
RAM is by far the biggest issue of the two, with a single 320x288x16bpp buffer using ~184kB of RAM, which is nearly 3/4 of the RP2040s 264kB -- 2 buffers simply won't fit.
The question then becomes: why not use smaller 160x144 buffers and then scale them up to 320x288 before? There are two very good(allegedly) reasons for not doing this:
1. Doing so requires additional CPU cycles to scale the image -- this eats into CPU budget that's needed for other duties
2. Scaling still requires an additional buffer that is at a minimum 320x144. This along with the two native size buffers total that same 184kB mentioned earlier -- not unreasonable, but not worth it when bundled with the CPU cost.
As such, the approach I've taken is a bit different, and gets me all of the scaling more-or-less for free -- as an added bonus, the implementation is very simple.
In the PIO program that ingests pixel data from the GBC CPU, I simply push each pixel out to the TX FIFO twice, effectively a 2x horizontal scale for free since there's additional time budget in the PIO. For vertical scaling, I can simply draw each line twice across two LCD lines -- or just leave every other line blank. This way I maintain only two buffers sized 320x144 for that same 184kB RAM cost, and get to skip most of the CPU time needed to scale traditionally.
Ill Tidings
With scaling figured out, you'd think the rest would be simple since we can already draw to the LCD. However, the big thing that I haven't gotten into yet is SPI bandwidth.
SPI is a serial communication protocol, which means it transmits a single bit per clock pulse. This means to transmit 60 frames that are 320x288x16bpp requires an SPI clock of a bare minimum of ~88MHz, with some additional to provide overhead for the SPI instructions needed to setup draw calls.
History tricked me into thinking this wouldn't be an issue. The Time Frog Color uses the same strategy, with some additional shenanigans to upscale by 1.5x to fill the 240x240 panel, but still for "free". The bandwidth required for that is ~50Mbp/s, which worked fine despite the ST7789 LCD driver IC being officially rated to only handle something like a 20MHz SPI clock. I was -- seemingly -- able to bump SPI clocks as high as 100MHz with no issues. Incredible.
The ST7796 in the Egg's panel is rated similarly, but initial testing indicated that I couldn't bump the SPI clock quite as high -- 50MHz seemed to be the ceiling. That's fine! I can just do that thing I mentioned earlier where I leave every other line blank, which cuts the necessary SPI clock to 44MHz -- barely in budget, but it works, and nerds love "scanlines", as it were.
Shit Hits the Fan
All of this was figured out back when I was still working with EBC-LCD-DEV -- as it should, as that was the whole point.
Sadly, things are not always as they seem, and it turns out I hadn't done sufficient testing before patting myself on the back and calling it a day. As soon as I assembled the initial motherboard and got it booting games, I noticed that things were not running at full framerate, like originally thought. Games played smooth... ish, but there was a lot of tearing. Tying screen updates to TE ending up hard locking the framerate to 30FPS, which was odd given my prior bandwidth calculations. No matter, I thought, I'll just play with SPI clocks to see if I can go a bit higher than my prior tests.
This is when shit got weird.
While testing out various SPI and RP2040 clock combinations, I noticed that there were a number of cases where higher SPI clocks performed substantially worse than lower clocks -- not unstable, mind you, just worse. This made me question whether or not the SPI clock I was configuring -- which was done through the TFT_eSPI config -- was actually the same as what was being generated.
I was basically tearing my hair out at this point, because it didn't make a lick of sense, and I was on the verge of pulling out the oscilloscope to start poking around. Then I made a breakthrough.
C++:
// Set clock divider, frequency is set up to 2% faster than specified, or next division down
uint16_t clock_div = 0.98 + clock_get_hz(clk_sys) / (clock_freq * 2.0); // 2 cycles per bit
sm_config_set_clkdiv(&c, clock_div);
This is the code in TFT_eSPI that sets the PIO clock divider based on the configured SPI clock. This piece of shit, for reasons that I haven't been able to piece together, is
intentionally truncating the result of this float calculation to an int, and the closer the system and SPI clocks are to one another, the the larger the possible error gets.
Some examples:
SPI(configured): 40MHz, SYS: 120MHz -> clock div of 2 -> SPI clock of 30MHz
SPI(configured): 50MHz, SYS: 200MHz -> clock div of 2 -> SPI clock of 50MHz
SPI(configured): 40MHz, SYS: 200MHz -> clock div of 3 -> SPI clock of 33.3MHz
SPI(configured: 50MHz, SYS: 150MHz -> clock div of 2 -> SPI clock of 37.5MHz
The first and last examples are especially egregious, with an error of -33.3% between configured and actual. This all explained why I wasn't getting consistent results.
C++:
//Set the state machine clock divider (from a floating point value) in a state machine configuration.
static void sm_config_set_clkdiv (pio_sm_config *c, float div)
This is the actual function definition from the RP2040 SDK, which quite explicitly accepts floating point dividers, so the rationale for TFT_eSPI doing what it does just completely escapes me.
C++:
// Set clock divider, frequency is set up to 2% faster than specified, or next division down
float clock_div = clock_get_hz(clk_sys) / (clock_freq * 2.0f); // 2 cycles per bit
This was the entire fix. As far as I can tell, this had no negative impact, at least in regards to my use case, and I was finally getting my expected SPI clocks. The day is saved.
Solving the Bandwidth Problem
Except not quite. Now that I had reliable SPI clocks, I found that the ST7796 does not in fact accept SPI clocks up to 50MHz. The actual cap was somewhere around ~40MHz, which is below what I need to get full frame rate updates, which means I needed to trim bandwidth usage somehow. The solution to this -- with some notable tradeoffs -- was to go all in on the concept of partial frame updates, i.e. only updating the changed parts of the image.
The ST7796 -- along with many other LCD drivers -- contains an internal frame buffer, and will maintain whatever image has been drawn to it without requiring a continuous flow of image data. This fact, along with the ability to address specific areas within the panel, is what enables these partial updates.
In short -- because this whole thing is getting far too long winded, even for me -- the way partial updates are achieved is by:
1. Breaking each row in the image into chunks of 16px in width
2. Compute a checksum for each chunk, and compare it to the checksum for the same chunk in the previous frame
3. If the checksums match, the chunks are the same, and that chunk does not need to be redrawn
4. If the checksums don't match, the chunks are different, and the new chunk should be drawn to the screen
5. Adjacent chunks that do not match the previous frame are joined into one larger chunk to save on SPI instruction overhead
This actually works pretty well in most cases. Game Boy games are very often:
1. Non-scrolling, and so the only changing parts of the image are sprites, or
2. Have scrolling but utilize simple backgrounds with lots of empty space
And as a result, on average less than half of the screen actually changes from one frame to the next. There are two main exceptions to this:
1. Full frame fades will usually trigger a full frame update, though these normally happen outside of active play.
2. Scrolling areas with unusually detailed backgrounds will very have a high enough change percentage that it pulls the framerate down. Examples are the overworld in Mario Land 2, or moving between screens in Link's Awakening.
Just today, I've been tinkering with ways reduce the impact this can have on game play -- where that's relevant -- if not actually reducing how obvious it is when it happens. The current method:
1. While rendering a frame, count the number of changed pixels based on the number and size of mismatched chunks.
2. At the end of the frame, if the number of changed pixels is over a certain threshold -- currently set to ~50% of the full screen pixel count-- flag that frame updates are behind
3. On subsequent frames, when the flag is set, only update every other line. Even lines on one frame, odd lines the next, then repeat.
4. At the end of a frame where the flag is set, if the number of changed pixels is below a certain threshold -- currently set to ~25% of full screen pixel count, because the running count only takes into account half of the screen -- turn off the flag and go back to updating every row on the next frame.
This is working out pretty well from a smoothness standpoint, but has the tradeoff that vertical scrolling sections, especially, will look a bit blocky, especially if it's scrolling at exactly 1 pixel per frame. The plan for the future is to add the ability to turn this feature off from the OSD in case some people find it distracting.
Long Term Solution Plan
Ultimately, I would like to not have to use partial frame updates, and so I've been looking around at different panels. I already have one of the same size that accepts an 8-bit parallel interface. I've also identified another that should be able to support QSPI, which would quadruple my bandwidth at the same SPI clock. Both of these solutions require more GPIO than I have on the RP2040, so they'll necessitate moving to the RP2354B for the expanded GPIO.
Other Things!
As mentioned at the beginning, the software is also responsible for controlling the brightness and volume levels. This is super simple stuff that doesn't require much explanation. Each control leverages the RP2040's PWM channels, with one GPIO each:
1. Brightness: controls an N-FET where the drain is connected to the cathode of the LCD backlight.
2. Volume: goes through a simple RC circuit before getting fed into the volume control pin of the audio amp, which is controlled via simple analog voltage level.
More fun than that is the OSD that controls all of this! It's all controlled by a single button on the side -- again, I have zero GPIO left at this point -- so it's an interesting challenge from a usability perspective.
Pressing the button brings up the OSD. Short pressing the button cycles through the different levels for the currently selected setting, i.e. brightness or volume. Long pressing swaps between the two settings. The OSD will disappear after 5 seconds with no input.
You might also notice the battery indicator, which is surprisingly functional. The system's VCC is connected to one of the ADC capable GPIO pins via a voltage divider to bring it under the 3.3V max rating. Battery level is displayed at one of five levels, based on an attempt to predict the current location in the discharge curve from the current voltage of VCC. I'm a silly man and forgot to add a cap to smooth out the value being read, so the value used for the displayed indicator is an average of the past 100 readings.
There's also very basic charge detection. Because the charge IC I'm using shunts VCC to VBUS when the device is plugged in, VCC will read above the max expected for a LiPo when this is the case, and that's how the detection is done. It can't, unfortunately, detect when charging is done -- this is yet another thing I would do with more GPIO, where I could just route the charge signal from the charge IC directly into the RP2040.
What's Next?
My submission video! The system is essentially complete with the final software changes done, and the milling of the missing OSD control button. I'll be working on getting my submission video filmed tomorrow morning, and uploaded just in time for the completion of the contest.
