This is kind of where I'm at. I don't have any "main-line" Star Wars sets yet, but I just ordered Luke's Landspeeder and the TIE Fighter to get a feel if I am interested or not. I guess I could set them up as display pieces, or just save them for my son in case he ever gets interested in SW after his Ninjago phase is over, but otherwise I won't be "playing" with them at all, and every dollar I spend on one of those sets takes away from a dollar I could spend on a modular building, or saving up to get some big sets that I missed, like the Green Grocer, Eiffel Tower or Taj Mahal.
I may just stick with the UCS sets (anything in the 10,000 number range), as they are fun and involved builds, make great display pieces, and usually increase in value the most.