The chlorine is a minor irritant although for me I think swimming is a net
positive

I have never had an actual infection from swimming though and it is not that
common - I know lots of swimmers.

A lot of pools are greatly overheated so you can look for one of those.

When I feel that I may be coming down with a sinus infection, or just for
insurance against sinus infection, I flood my sinuses, with my forehead
pointed down, with a mixture of 3% hydrogen peroxide, baking soda and kosher
salt.  I dilute the 3% peroxide by adding 15 parts water to one part
peroxide, but it still stings quite a bit.

My ENT advised me to use a snorkel mask, one that covers the eyes and nose.
He told me that he himself uses one as he has sinus problems from the
chlorine.

I've been using one for a few years and the sinus problems from swimming
ended, but I don't sit around a chlorine fume-filled indoor pool with no
mask on either!  I've swum in one pool that uses saline instead of chlorine
and had no problem there without a mask.  (It's a bit like irrigating if one
swims badly :-)   )

I also avoid diving or swimming underwater as the pressure even a few feet
under causes problems for me.

George