Non-custodial software generates and stores keys on your device. Transactions are signed locally, and the provider, if any, never has the ability to move your funds.
This removes counterparty risk and replaces it with operational risk. A lost seed-phrase, a signed malicious approval, or a moment of carelessness with a wallet-drainer produces exactly the same outcome as an exchange failure, only faster.
Most experienced users run several: a hardware-wallet for long-term holdings, a hot-wallet for routine activity, and a burner-wallet for interacting with anything new or unaudited.
Related: custodial-wallet, self-custody, hardware-wallet, burner-wallet