Good question (even if I'm actually the one asking it). First, recall that it's incomplete to say that everything has potential. Everything is potential itself, it's not an attribute that we have but the very substance of what we are, and it's only the human mind (maybe some other mammals, too, but we can't really know because of that whole inter-species communication thing) that separates the unified field of potential into separate objects and things, including "self'" and "other."
This includes the concept of potential itself. Whatever it is that we think of as potential is actually just another thought, itself a separation by our minds from the whole of the rest of the unified field. If we think that potential is x, then the rest of the universe, everything that isn't our concept of potential, is logically something greater than x (x + y, where y is everything that we've decided potential is not). In other words, we can never grasp potential intellectually - whatever we think it is, by definition, it isn't.
If you decide that potential is in fact x + y - your concept of potential and everything that's not your concept - then that implies the existence of some z, as x and y can be summed into some single totality and z is everything beyond the concept of that totality, and so on down the rabbit hole of ideation and mental constructs. Potential is beyond what our minds can conceive - it is literally the inconceivable.
But when we quiet the mind by sitting still and concentrating on not engaging our thoughts, the conscious mind, or at least that constant running monologue in our heads, starts to quiet down. We can just sit (as opposed to sitting and observing, or sitting and identifying our moods or our states of consciousness), and when we just sit, the cookie-cutter mind quiets down and stops dividing potential into this and that and we can actually start to experience potential instead of merely objectifying or trying to conceive of it.
If during this experience the thought arises, "Aha! So this is what potential must feel like" (or something along those lines), then we've just lost whatever it is we were experiencing, as the very thought itself has just separated the observer from the observed. There's no point in sitting with the intention of manifesting potential, to actively seek out the experience of potential, but if we instead just sort of give up trying and engage in a sort of goalless "just sitting," the experience will manifest on its own.
Even if we can't grasp it at the time, afterwards we are subtly different for having directly experienced that each of us are not separate from the fabric of potential but are a part, for lack of a better word, of the unified, interdependent, and impermanent whole.
Don't want to experience potential? Fine, your choice, then don't meditate. Don't want to meditate? Again, your choice, but then we can't you and I have an informed conversation about potential. When it gets down to it, it's really all just a matter of personal preference.