Dag makes the valid point that complaints by yeshiva parents that they are being "double taxed" is erroneous. I, however, would like to expand on his comments.
The expense of running public schools is a "burden" that the entire community pays for - not just those with children. Those who have children, those who never had children and all those whose children have long since grown up pay "tuition" to the public school system.
There is a valid reason for this - it is in the public's best interest to have the populace educated. It's not just a benefit to the parent - it's a benefit to society at large, since an educated populace is able to produce a better society than an ignorant one.
To state that people are "double taxed" because they are paying tution to private school is akin to someone saying that they don't want to pay "police taxes" because they have a private security guard.
Lastly, for all those parents who want to allow for opting out of payment for public services because they aren't using them, I would like to ask this question: Would you allow for someone to opt-out of paying for services for fire and ambulance because they were fortunate enough to not need them during the past year? Because doing so would lead to a situation where you would have to pay on a case-by-case basis for payment of these services. Would you want firemen to show up to your burning house and present you with a bill? Or would you like for the public ambulance service to ask you for your credit card number while they're taking a loved one to the hospital?
The bottom line is that we all benefit from these services even if we don't actively use them. For example: you may not need to call the police this year, but because the police were there when someone else called, a mugger who may have mugged you is now in prison. Likewise, you may not actively send your kids to public school (heck, you may not even have kids), but you use products and services created by public-school educated people. Your boss, co-workers or employees will probably be public-school educated people. Without the public school system, our economy and society would be markedly different from the way it is now - for the worse. As a result, we all benefit from the public schools and cannot shirk our civic responsibility to pay for these services even if we don't actually send our kids there.