Our perception of things as 'sexy' revolves entirely around our senses. Some of those senses are a little less commonly spoke of, and some of them are creations of man as subsenses. The most commonly perceived 'Sexy Sense' would be sight. Most people assume that finding someone sexy is completely shallow, when it is in fact the entire opposite in quite a few cases. Something about what they wear may catch our eye. A cool colors blended nicely with a warm color, giving you a calm but excited feeling throughout. This subsense is often referred to as a 'sense of style'.
Other senses play large roles as well, but society doesn't put nearly as much emphasis on them. The sense of taste can amplify our opinion on how sexy someone is, but is often overlooked due to the obvious implications. Explicit acts or not, one cannot deny the power of the sense of taste. Our sense of hearing is another often overlooked sense that leads us to the perception of sexy. Someone's voice may be very deep, rumbly. Such a strong voice, filled with an even stronger bass, may set your body to flames. Alternatively, it can soothe you, which leads to comfort and an inadvertent attraction. Thus, our sense of hearing will also cause us to think someone is sexy.
Our sense of touch works mostly the same. Some prefer smooth skin to rough, and vice versa. Smooth skin is often attractive and sexy to those with rough or calloused hands, as it's a feeling they aren't used to. It's new and different, and it amplifies the already effective sense of touch. With this logic, those with soft hands often prefer rough, calloused hands. This is where the idea that smooth women and rough men make great partners. There are other things that cause us to be attracted to rough hands, but that's a point for a much more explicit subforum.
Going with all of the information above, it can only be assumed that the sense of smell also plays a large role in our perception of sexiness. Perfume companies that sell their products to women aren't creating things that women enjoy smelling. Men are attracted to the often time feminine and fruity scents that perfume carries. Colognes, as well, are created with the intention of attracting women. This is why people mainly use perfume and cologne when going on dates, or when they wish to impress their fellow employees at a work event. Don't worry about thinking that some colognes are sexy, because they are created with just that intention!
Other scents can be perceived as sexy as well. Natural scents, such as what you'd smell like after a day in the woods, can also be sexy. The smell of sweat can be attractive to some. Note that there are different types of sweat, and each has a different scent. Stress sweat, heat sweat, work sweat. Least attractive to most, left to right.
If you have any other questions about these sorts of things, just let me know. I love talking about this stuff!