Science is always changing but the Bible stays the same.

This is something i often hear about science when it contradicts a specific interpretation of the Bible and is deployed as a criticism as if it is a weakness rather than a strength.

Can someone who feels this way explain to me why it is a BAD thing to change one's position on the basis of new evidence and a GOOD thing to hold to a position despite contradictory evidence?