Using the component cable and AV mode (composite) on the TV I can get the picture to appear in black and white. When I switch to 480p mode in the wii menu, I get duplicate screens side by side in AV mode. WHen I switch over to component mode on the TV, its black with no signal...
So now the troubleshooting... Here's what I've figured out so far...
I was able to test the cables on a Wii U and they seemed to work fine.
It sounds like your TV has combined CVBS / YPbPr sockets. On some capture cards, these are strangely combined in the way that the composite video plug inserts into the Pb socket, not into the Y socket. If it's implemented like this on your TV, it would cause exactly this issue that you are having if you are not re-plugging the cables after switching from one input mode to the other.
Then again, I assume that when you tested the cables on your Wii U, you did make sure to plug them in in the way that the TV expects them, so I think it's quite unlikely that you only forgot to replug them currently when using the Wii. But still, stupid mistakes just happen sometimes, and from your description, it really does sound like more of a TV cable wiring issue than a Wii video output issue. You have confirmed that the Wii is indeed outputting 480p over YPbPr, otherwise you wouldn't be able to see the double frame picture on your TV when having it set to interpret it as CVBS. So I really doubt that your AVE chip is to blame.
On another note, I personally don't solder the mode pin to 3.3v directly, but both of these to a switch, so full YPbPr and CVBS output is retained alongside VGA, even with Priiloader VGA installed. Might be helpful for troubleshooting and testing VGA output while you keep your mainboard still fully intact with the Multi AV output plug and everything.
For CVBS: Set the switch to off and boot the Wii with either composite cables or no cables connected
For YPbPr: Set the switch to off and boot the Wii without cables, connect the component cables after 10 seconds of startup or so.
For VGA: Set the switch to on