CentOS - Jul 2017 - firewalld: whitelisting/blacklisting addresses allowed to connect to a service/port with ipset

I'm trying to figure out how to use firewalld on CentOS 7 to block access 
to ssh (on a custom port to control log bloat) and smtp submission except 
for specific source addresses, using ipset. I haven't been able to figure 
out how to combine a port number or service name with an ipset, either as a 
blacklist of nets or a whitelist of addresses. It looks like ipset with 
type of "hash:net,port" might work but the current version of
firewalld on
C7 doesn't support that type. I fear I'm going to have to write a direct
rule. Has anyone combined ipset with a port to achieve this? I tried a rich 
rule but I can't specify both an ipset and a port as the source value. 

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus