Imran refuses to meet ‘defectors’