I look for three things:
1) Does it catch my eye? This can be anything from bright colors to an animation, to just plain looking cool in general. I don't pay attention if nothing is interesting enough to draw my gaze in the first place. The best way to do this is catchy text--instead of just the RP title, a witty or interesting catchphrase is good.
2) Is it well made? If it looks like someone put effort and thought into the banner, they probably put effort and thought into the RP itself.
3) Does it look like something I'd be interested in? If it's obviously a furry RP, for example, I won't click it no matter how well made it is. But if it's Fantasy or Scifi I'm likely to.
I tried to take these into consideration in the creation of the banner for CYHM. While a friend volunteered to actually make it because I have no art skills, I selected the background and text. The catchphrase is vague enough to make you want to know more, the background makes it clear it's some kind of Scifi, and it has a lot of details like the binary and "echoing" text. While I don't claim to have actual advertising experience or anything, my family runs a small business and has designed plenty of logos, so I've picked up bits of knowledge here and there.