Mustafa Nawaz Khokhar refuses to take ministry