Well you need to gather more hints from the other NPCs to narrow it down.
Sometimes you wont get the perfect hints and end up with a lower chance to get the correct hole.
For example: You would need 3 hints to know the correct hole.
North, West, Human.
The correct hole would be top left next to an NPC.
If youre getting the hints North and Human you would have a 50% chance to get the correct hole as its either top left or right.
As you see it all comes down to the hints youre getting so make sure to gather as many as possible.
Hope that helped you out.