According to Adobe, you cannot remove permissions from PDF files without being an administrator on the file. The administrator has full control and determines the permission settings.