If you are happy with how it looks and its structurally fine I would go to the planning office and ask if they will sign off the change.
If they do then your all good and covered if any neighbour complains
If the planning office decline, then get the builder to rectify the issue. Its better than taking the risk and having to pay out of pocket to fix it