There are four reasons why is it possible to take refuge in the Three Jewels.
First of all, the Buddha himself is free of samsara. We cannot protect others because we are still stuck in samsara. The analogy is that of a double drowning: if we do not know how to swim, how could we save others?
The second reason is that the Buddha has the perfect skill of being able to free others from samsara.
Thirdly, he has great impartial compassion, not seeing some as close to him and others as distant.
And finally, the Buddha benefits everyone, regardless of whether they have helped or harmed him.
~ 17th Karmapa: The Torch of Certainty, Session 1