Menus > Layers > Delete > Unused Layers

Layers > Delete > Unused Layers

Deletes all layers with no objects on them.