In my own experience I would strongly disagree. I use reference all the time as practice and if anything it allows me to understand the figure better so that when I draw from imagination I have real structure to draw upon.
Take 15 minutes to practice every single day. Find the time. Schedule it into your daily routine :)
