As a less visibly disabled person, you can choose whether or not to disclose your disability. There will (usually) be less judgment if people don't know that you, in fact, have a disability. As a visibly disabled person, I have no choice, but to disclose, because the sheer fact that I walk in with crutches or roll in using my chair sends a big red flag of disability up. Because of that, I am treated as less competent or patronized. I have no choice in the matter. Before I meet new people, I am plagued with insecurities about how they will react when they see my mobility aids. I wonder if those who are relatives or friends of my parents know I am crippled, if they have been told. I wonder if a disabled person fits their mental image of what I look like, who I am. I don't have the luxury of being able to choose whether or not to tell them, to be able to hide it.
As an invisibly disabled person, you can blend in, succumb to wonderful anonymity. Every person on my college campus knows me, not by my name, but as "the girl with the scooter". I constantly feel like there's a harsh, bright spotlight on my head that never, ever goes away. Even on those days when all I want to do is curl into a ball and disappear, I can't. Because people notice me, and I really, really wish they didn't. And for the record, all the speed limit jokes get old. Really old, like nasty-expired-food old. Those oh-so-witty jokes are tossed my way whether I want them or not.
So, yes, you over there with the "invisible" learning disability, or other not as visible conditions, you have privilege. It doesn't make you a bad person, just as having any other type of privilege (white privilege, class privilege, cis privilege, straight privilege, etc.) does not make you a bad person. It's just something you have and need to acknowledge. The fact of the matter is that I can't ever be a wolf in sheep's clothing. I am simply, inexcusably, the wolf, claws and all.