Who have logs of how users vote?

Besides it's a very important question for our privacy, I suspect some of my comments getting downvoted at set periods of time from other accounts. I don't understand how the fediverse work and what I'm asking is who has vote logs? Who I can ask, if a bot, like on reddit, got dedicated to downvote every post from one user. How to avoid vote brigading on here? Can it be automated?

I'm not pissed and I do know I post literal shit sometimes. It's just I'm wondering, if our fedieverse have machanics against that.