We are on clay. Lot depends on how much garden you have. I would be tempted to use an excavator to break up the top 12 inches, remove any junk, and mix in a huge quantity of organic matter. Perhaps a 50:50 mix. We only did this on the beds but wish I had done something similar for the lawn. We have a land drain across it but its too wet in winter and we get a lot of moss.
I was warned off sand because apparently it can turn the clay into cement. Not sure if that's true for all clay.