You are correct in your understanding that anything in the Core code will be overwritten during an upgrade. In fact, it’s safe to assume that anything that comes with the default Magento download has the right to be overwritten. This includes templates.
The safest way to alter Magento is the following:
1) Altering code: Create a module that extends the current Core functionality. This will keep it separate from the core code and make it persistent during the upgrade. The only issue you may run into is, if extending core code, you may need to make some adjustments to your code after the upgrade. I haven’t had to do this yet though.
2) Altering the templates: Copy the template you’d like to modify and paste it under a new name (all folders and files under app/design). Then modify your copy. Varien updates their default template a lot, so your changes will always be overwritten. Best to create a new template with your changes.
Does this answer your question?