Do you motice the drifting in your prints theirselves? I would try printing a 4'x8' "checkerboard" of exactly 1' squares, then check your results to see how it is skewing.
I am suspecting your linear encoder strip is dirty, although when it goes wonky, it is usually VERY noticeably...