Hey Robert, there is a way to do this. In the group_useradded.xml template you can add the following logic to the subject field.
#if($core_v2_group.Current.Id == <Group_Id_to_ignore>)
$email_utility.CancelEmail()
CancelEmail() will prevent the email from being sent. You can add that logic for as many groups as necessary.