PM trying to uplift poor: minister