The View menu > Previous Zoom command takes you back to your immediately previous zoom settings. If you use the View menu > Previous Zoom command repeatedly without using other commands to change the zoom in between then the effect is to toggle between two zoom settings. You can not use the View menu > Previous Zoom to go back more than one zoom setting.
The View menu > Previous Zoom command has no effect in the following circumstances:
Immediately after you first display a view in a window.
Immediately after you use the View menu > Restore Full View command.