Menus > Layers > Undo assign

Layers > Undo assign

Undo assign moves the last assigned objects back to their previous layer.