Here's how to get SMAA working in this game.
1. Download the SweetFX 1.5.1 with x64 support from this page:
http://sfx.thelazy.net/downloads/
2. Extract all files into the game's x64 directory which is where the executable is located. (C:\Steam\SteamApps\common\ShadowOfMordor\x64 - replace my Steam path with yours)
3. Enable SMAA and anything else you want in the SweetFX_settings.txt file
4. (The part no one explained) Move the DLL files out of the X64_dll's folder that got copied in with the rest of SweetFX, and put them into the game's x64 directory where the executable is, overwriting the 32-bit DLLs.
5. No more jaggiessss!!
Add to OP?