Is it ok to drink a glass of water that's been left out all night?

In our house, a glass left out that long starts to taste a little different. I call the taste "dusty," and I prefer water straight out of the tap. Your conditions may be different. I often re-use a water glass, but I rinse it a bit each time.