Groups including Save the Children say move would damage Britain’s international standing