Uncheck the "scale with image" button in the object properties tab.
If you don't have that docker up it's Window>Dockers>Properties second tab from the left (outline).
Gene
or
1. select the item in question
2. Click on the text tool
3. Click on the 'Outline pen dialogue' or f12 key
4. Check the 'Scale with image' box
5. press ok... you're done
BTW if you use contour, every time you scale it has to compute a new contour which can be very time consuming on complex objects.