I have a cat

Just adopted a 5 yo cat. Before he was living in a family of 5 peeps. When they brought him the first time he was scared to death all he is doing is hiding I also noticed some black dots coming out from him, his hand was injured I didn’t understand at first, he was infested with flea and he was irritated all the time.
Now he is still hiding he only gets out to eat or to go to the bathroom and only when I am not around.
I don’t know what should I do really I feel bad for him, one thing I am sure about he was not well treated in the former family he was living with.