I feel like people depend on me too much at work

I always get treated like I'm in charge of everyone. I literally have no clue what I'm doing half the time at the workplace so it's like I don't like giving out orders. It's hard enough like just being there but then I feel like I have to watch over people and I don't want that responsibility. I just want to be able to do my job and not worry about what everyone else is doing