I decided to have a try with the amp powered from another source and disconnecting the controller after I got into a game, so the only connections to the mobo were the screen and everdrive. Still got noise but even more strangely the audio had another issue, the balance of dialogue and music was well out of whack. In menu for Mario the "It's me Mario" and "Press start to play" are barely audible though the music is still in stereo. I'm powering the amp from a usb on my pc reading the exact same voltage as I was getting of the N64, very strange indeed, the fault finding continues.
Edit: just tried the PAM amp from the BOM, same audio issue there too.