The difficulty comes largely with glasses, not contact lenses.  It has
three forms.

One, is the image size differences between the eyes.  2D to 4D is
likely tolerable and even more if long standing and the person is
adapted.  Depending on whether the power difference is due to a
difference in axial length or corneal curvature of the eyes, this
problems may also exist with contact lens wear, but is rare with
contact lenses.

Two, relates to different prism effects induced in each eye when
looking away from the optical centre.  This can be largely eliminated
by the person learning to turn their head, not their eyes when
looking.  This is a problem only with glasses, not contacts.

Three, applying only to bifocals or multifocals, relates to vertical
prism induced when looking down to read.  This can be made tolerable
by special grinding of the lens during manufacture. This is a problem
only with glasses.

If the two eyes have quite different best corrected acuity (even if
the Rx in each eye is the same) due to disorders like cataract,
macular disease or amblyopia, then vision with both eye looking may be
reduced compared to that of the better eye due to processing problems
in the brain.  This problem is equally found with glasses or contacts.

Dr Judy