The actor that played Q felt his character had gone as far as it could. He decided to end his time after season 4. Yes, it was difficult to see him go since the series was basically his quest but season 5 was still wonderful without him.