I certainly cannot determine exactly what is going on but there are a few general principles.
Is sounds like you are experiencing what we call "foreign body sensation" which occurs when there are tiny abrasions on the cornea. Alternatively, this can be caused by an inturned eyelash or a stye.
A soft contact lens is an effective barrier between the blinking eyelids and the irritated cornea and in the presence of small abrasions, the eye is sometimes much more comfortable with a contact lens in place.
That being said, we strongly discourage the wearing of contact lenses when there is any irritation of the eye. Indeed, a contact lens is generally clean but not sterile and could introduce bacteria or fungus to an abraded cornea causing a serious sight threatening infection. So, if any irritation, don't even think about inserting a contact lens. It may be that the contact lenses are causing the irritation and it is only when you remove them that you actually discover how abraded the cornea is.
Whatever the problem, dryness or contact lens related irritation, it needs to be diagnosed and treated before returning to your contact lenses.