What is wrong with material enjoyment?

Renunciation or abnegation for ultimate good is certainly a better occupation than enjoyment in the diseased condition of life. Such enjoyment only aggravates the symptoms of disease and increases its duration.

Srimad Bhagavatam 1.2.6 purport