I used to be a total eraser hog and take FOREVER to complete a drawing, even a quick sketch. I've gotten a lot better recently, and during my LoA sessions I intentionally "forget" to bring an eraser so I force myself to commit to whatever I do. In the quicker drawings it allows me to really easily see where I went wrong, and I find I am able to give myself better feedback.
I recently have been letting my eraser sneak back in for the longest drawings of the session, only to remove small detail mistakes, but I'm not sure if it's helping my technique or not yet.
Most of my instructors over the years have had a no eraser on exercises policy, for exactly the reasons you described - it shows you where you went wrong, and helps to teach you not to second guess and slow yourself down. Exercises are, after all, just that. It's okay if the mistakes stay in them. :)
I've had numerous instructors say something along the lines of "(s)he who erases most, wins" when it comes to work that's going to be "finalized." Of course, a lot of my instruction was in scientific illustration...