"One's rights end where another's begin" - Morally speaking I agree with this, and I've heard this phrase used by animal rights activists to argue that humans shouldn't have the right to violate animals' (moral) rights to be free, to not be killed, harmed, exploited etc. at least by humans who are moral agents & don't need to do so.

Again, there is a difference between moral and legal rights. Just like in the case of human slavery where some humans technically had the legal right to enslave other humans - and I would agree that those laws were unethical to begin with since the moral rights of those slave owners to do things ("positive" rights) ended where the moral rights of the victims to be free from oppression/harm/etc ("negative" rights) began - many people argue that the current legal rights of humans to, basically, enslave & kill non-human animals, are similarly built on unethical laws, and don't translate to moral rights, in the sense that humans' rights also end where other animals' rights begin, morally speaking (such a position would of course entail action to liberate non-human animals via boycotting of animal exploitation (veganism) as a moral obligation, similarly to how when the laws that enabled people to own slaves were in place, boycotting the slave trade and being an abolitionist would also be considered a moral obligation by most people today).