Fees for widely held large caps are often a few basis points a year. For a heavily shorted small cap with little available float, rates of 20%, 50% or even several hundred percent a year have occurred, which can exceed any plausible gain from the short.
The fee is variable, not fixed at the outset. A borrow taken at 3% can reprice to 40% if demand spikes, turning a viable trade into a losing one without the share price moving at all. This repricing risk is a material part of what makes shorting difficult.
Because the fee is quoted annually but charged daily, a 60% borrow on a $50,000 position costs roughly $82 per day. Model it explicitly in any short thesis, alongside the risk of recall. See hard-to-borrow and short-rebate.
Related: hard-to-borrow, short-rebate, securities-lending, short-selling, short-interest, locate-requirement