Adding and removing multiple IP address to AWS security group using Ansible

I was trying to use Ansible to add IP addresses to an AWS security group.

I came up with a task syntax that looks like this:

- hosts: localhost
  gather_facts: False
  vars:
    ip_addresses:
      - 1.2.3.4/32
      - 2.3.4.5/32
  tasks:
    - ec2_group:
        name: security-group-name
        description: Security group description
        vpc_id: vpc-1234567
        region: us-east-1
        profile: profile-name
        purge_rules: false
        rules:
         - proto: tcp
           from_port: 123
           to_port: 123
           cidr_ip: "{{ item }}"
      with_items: ip_addresses

This does not do exactly what I was looking for as it basically runs the ec2_group task multiple times instead of just looping over the rules.

This also does not work if I set the purge_rules to true as then it will purge all existing rules on each iteration, effectively removing all but the last IP address on the list.

I'm wondering if there is something similar to with_items that I can apply to the rules attribute to provide it a list of IP addresses but calling ec2_task only once?