I'm guessing here, but maybe someone has already undone/done up this nut with a hammer and chisel? That distorts them enough to prevent a socket from fitting subsequently.
So either remove the damage enough to use a socket, or hammer and chisel if off and fit a new one.
Sorry to say that doesn't look good though. This nut is hardly an unusual enough size to require this butchery, but sadly it's common on old British bikes where any bodger thinks he can rebuild them using a toffee hammer and stillsons.