Will optimisation impact performance?

No. We use Azure Monitor metrics to ensure all rightsizing maintains or improves performance.