I'm using project64 2.3.2. I downloaded the release from
https://github.com/gonetz/GLideN64/releases and put the dll files from the zilmar-spec folder into project64/plugin/gfx. You should have the gliden64.dll, a custom per-game config file (GlideN64.custom), and your configuration file (named just GlideN64). I think the other files are just language packs. If you don't have a config file, it should be created after you open the settings.
If it won't save your settings, check for that config file and open it with notepad. If nothing else, you should be able to manually enter your settings there.
----------------------------------
Also, as long as this thread is being bumped, Rouge Squadron is now fully playable in HLE mode. I never tried the game myself, but I know it's been a tricky one to emulate:
http://gliden64.blogspot.com/2017/08/star-wars-rogue-squadron-hle-finished.html