Zune (1st gen) sometimes not starting when charged

My 1st generation Zune has been acting oddly for the past couple of months. It sometimes refuses to start (light the LCD) when it is charged (sometimes fully charged). Plugging the Zune into a power source (AC adapter in my case) lights the LCD and lets me use the Zune. But, still, it is annoying. It is a good thing I never take the Zune out? Otherwise, I might be faced with a temporary brick.