The solution in this case was to turn Inheritence OFF for that group and then withdraw the Firewall policy from that group. After the policy updates it should release the old on Windows Firewall. I didn't have time to stick around for that so we forced the policy update from the client and rebooted the machines for good measure.
1. Uncheck "Inherit policies and settings from parent Group xyz"
2. Click Tasks to the right of "Firewall policy" and Withdraw the policy.
Everything seems to work right afterward. Aside from the inconvenience of having a non-inherited policy to deal with later on when you want to make changes. In most cases I've found that NTP works a lot better than the older versions like 10.x had so you most likely won't ever need the contents of this post but just in case, have fun.