For the life of me I can not get the text and images to stop coming up. I turn them off in actions, then as soon as I come back to the set, the text & images are turned back on.
How do I make it so that no matter what sets I use, I never see text or images?
You can't exactly do that, but you can make individual sets do exactly what you want them to do, and stay that way.
The problem here is that PixelToy's "Sets" window isn't easily comparable to anything else in another program. Clicking on a particular set loads all of its settings, but any changes you then make will be lost the moment you load another set, without any warning. If you make some changes you want to keep, you must "Add this set..." from the Sets menu. You can replace an existing set if you want.
You can keep entirely separate Sets files, as you have surely noticed the "Other Sets" folder in the PixelToy folder. There is no facility to create new Sets files in PixelToy, you must duplicate an existing Sets file in Finder, then double-click it to use it. In PixelToy, any add/delete/rename set change instantly modifies the currently open Sets on disk.
I definitely appreciate that this is not at all intuitive! It sprang from the demands of people doing live performances, who need to preset certain visuals, then tweak them live, and then switch to completely different visuals without being asked to save their changes.