making sure that EIP is not reused across VPCs

Given:
One (ore more) elastic IPs that we want to reuse. (the ip pool)
A CloudFormation template with a machine that has an elastic ip associated to it. (via AWS::EC2::EIPAssociation) Bringing up / tearing down multiple CloudFormation stacks that get ips from the shared EIP pool. (the ip is a parameter in the stack and is computed before the the stack is launched)

My expectations were that if one VPC / CloudFormation stack is already using an EIP, other attempts to use it would fail.
It turns out that if you attempt to use the same EIP in another stack, it will work. The EIP will be bound to the latest stack that is created.

To make it ever worse, if say stack 1 uses an EIP, stack 2 uses the same EIP, if you tear down stack 1 after stack 2 is in use an 'stole' the IP, stack 2 will lose the IP (resulting in the IP being free - i.e. not bound to any instance).

Has anyone run across this issue? How do you ensure that you're not running in situations above?

The reason I have to use the ips is that they are whitelisted for access in a system I'm interacting with over the public internet. The traffic must come from the IPs in that pool and cannot just grab any random public ip.