Is it okay to turn down your boss from doing a task that's not really part of your job description?

The task would be time consuming, more responsibility and can take some time from your regular day to day job.