It’s the natural type that gets oily, the other types don’t separate like that . You could pour the oil off but then you’ll have a dry-textured peanut butter.
Peanut butter is an emulsion of water in fats, oils, and peanut solids. "Natural" and nearly-natural peanut butters contain very little else, so they flow readily but also readily separate into oil and solids, and their volatile oils may evaporate after you open the can lid .Best is to keep it sealed and shake it up well. Else ,use up the butter as fast as you can since it has a low shelf life.