Of course, you could play N64 games in an emulator and use whatever controller ... work on an add-on board that let’s you use Xbox 360 wireless controllers on Nintendo’s 1996 console.
These are the button mappings for an Xbox 360 controller. The emulator is a JavaScript Game Boy emulator. This is loaded on the Chromecast through a simple html file (called the receiver in the repo).
You'll probably need a game controller for certain titles ... all of the pros and cons of playing video games on an emulator. Many Xbox 360 games work almost perfectly, as if they were native ...