I must have done dozens of these in the past, there is a triumph tool, a double diameter drift to knock them out,they are not tight.
Use either bit of decent tube that will just fit into the housing and bear on the remaining bit, or a solid bit of brass or alloy.
Before you reassemble anything study a parts list for the top and bottom washers, so often these ave been left out because previous owners did not understand their purpose.
I repeat its not that tight unless its been glued in, your drift must be tight in the yoke ID