Quote: "Would it be a good idea to use the delete image and delete audio LUA commands before loading the images and audio?"
as long as you're checking they actually exist first sure, might get some errors if not.
Quote: "Does this at all affect performance"
not really in a positive way, it's possible you could get slow down if you were near the memory cap but not quite reaching an error state... but it's likely to cause slightly longer load times if you have to load large files and if you are doing this repeatedly then it is generally better to keep them
Quote: "or help prevent the memory from getting capped?"
during the level yes it can help... it's generally a bad idea to repeatedly load and delete a resource that is used frequently (for instance a map or inventory is probably ok to keep the bulk of as you are likely to want to keep reusing the backgrounds etc, to reduce space you can delete the smaller items that will change more often - as items get used during the game etc) but if you had an image at the start of the level that you no longer need for the rest of the level it's generally a good idea to delete it (e.g. the instructions for how to play when first entering the game or an intro sequence).
i'm pretty sure GG loads all sound at the start of the level so i'm assuming you're talking about global sounds here? anyway the same logic can be used for sounds - repeated (e.g. gun shots) = keep, low usage (e.g. intro dialogue) = delete
remember GG should already clear up anything between levels so this is only necessary if you are reaching a memory cap during your level.