Former gentoo user here. Compiling everything yourself does not magically improve performance. You have to use keep track of USEFLAGS, ideally cherry picking for some package because some can cause bugs or performance regressions.
It can be really time consuming both compiling gentoo and trying different configurations. (But you'll learn a lot of compilation/ build system knowledge along the way)
My advise is that if you have time and want to experiment and learn, sure go with gentoo. If not and performance is absolutely critical then go with Clear Linux, otherwise take your popular distro of choice, package availability and ease of use are more important than a couple of % in performance improvement IMHO.
We're talking about Qualcomm here, the company that made a deal with Microsoft to make Windows on ARM exclusive to Qualcomm SoC.