Caretakers denotified after PM’s oath taking