Jamie Dornan admits dealing With ‘Fifty Shades’ critics was ‘Difficult’