If your soil has poor drainage but not a high water table, and if inspection of your soil pit reveals soil that seems amenable to cultivation, then dig in plenty of organic matter.
Organic matter is beneficial to clay soils as it helps chemically to break up the clay into smaller crumbs. Adding sand, grit and gravel is seldom effective as it is not often practical to add enough to reduce the overwhelming influence of the clay content.
You will need one barrow load of organic matter per square metre of soil in order to make a significant difference to the soil structure and drainage.
It is still wise to select plants that are tolerant of wet soil conditions. Planting trees and shrubs on shallow raised mounds helps to protect them from waterlogging in wet soil. Many plants can be planted in raised beds to improve drainage.
Where there is a high water table, you can either decide to go with your soil type and plant a bog garden, or else consider installing artificial drainage systems.