Luckily, there is an easy way to get the chlorine out of your water. Just let it sit on the counter for a few hours. The chlorine will naturally dissipate (into the air as a chlorine gas.)
If you prefer to drink chilled water, you can let the water sit, then pop it back into the refrigerator. I prefer room temperature water, so I just leave my glass sitting out all day long and drink from it when I'm thirsty.
Unfortunately, this method does not remove fluoride or other harmful chemicals/toxins. You can use a filter to remove contaminants, but only reverse osmosis and distillation will remove fluoride.
In order to remove the chlorine from your bathwater, you would need to set up a whole-house filtration system or let your bath water sit until it's cold...brr!