Well that is indeed the trade off. The IoT chipset does get long support, but it is not up on par with specialized phone chipsets. If you are looking for a powerful phone with a snappy experience, my impression is, the fairphone would probably not life up to that. It is a phone for people looking for repairability as the main feature.
Fairphone ist your answer. It got a maximum score from iFixit and you can order and replace all components easily. They also promise support until 2031, even picked an IoT Chipset for longer support.