Can someone port my number without permission?

No. Porting requires your approval and passing biometrics before even start the process. This is to reduce the risk of SIM swap fraud.