N.Korea could deal ‘fatal blow’ to Olympics: S.Korea minister