Parliamentary committee facing difficulties in getting govt houses vacated