If by "drift" you mean to say which side will the finished product tend to be pulled towards? Your presumption is correct IF you are facing the infeed table, and not facing the out feed table [or like looking from the rear of the machine toward the front].
The tightest gripping side of any pinch roll situation becomes the pivot point, and therefore will cause any substrate to "ROTATE" on that point while the material is being fed through.