I'm not sure which is more damaging: homes becoming holiday-lets as you say, or their owners' second-homes - a problem in my part of the world.
Other effects:
Direct tourist developments including hotels, static-caravan sites, golf-courses and large car-parks destroying the scenery that attracts the tourists in the first place. (So do big camping and touring-caravan sites, but those are occupied temporarily.)
Cheapskate "attractions" like fun-fairs and amusement-arcades.
Tourist trades offering only largely-seasonal work on poor pay and conditions.
Over-crowding of town-centres, beaches, etc.; very heavy road traffic.
Heavy strains on the utilities and other public services.
Ironically many tourist "honey-pots" are in very deprived areas. For example, the County of Cornwall, in SW England, is all very picturesque and does have wild, beautiful scenery, fills with tourists and second-home owners (the latter skying house prices beyond local salaries), but is one of the poorest areas of Britain.