So what are you thinking about setting? There are so many possibilities, I'll just start asking questions curious about what you are envisioning
The Port, is this a small mostly self sustaining community, a minor stop along a minor/major travel route, an important stop or within range of a major trade route?
Population: Small like 80 people, like a few hundred, small city several thousand, a bustling hub of commerce. You said Island so I imagined it being small, but there are so many versions of "small" (and it could be big and full, Manhattan is an island...)
Oh that brings my thoughts to time period. Are we both thinking 'classic' with wood boats?
And as always the geographical type. The Emerald Isle is a great deal different than the Ring of Fire. Are you thinking warm or cold water?
Its the island mountainous, cliff faces, smoking volcano, beaches of rocks or sand or both, rainforests/jungle, bogs and marshes, barren rocky plain.
Weather? Rainy and dull, constantly sunny, harsh storms, best surfing waves, run like clockwork or unpredictable to all but those who know how to read the signs?
etc.
Guess those are mostly above water questions, but some of them can go deeper like the geography.