When I was just starting out as a solitary practitioner, there were few Pagan/Witch goods shops around; the majority of the places that sold the materials I felt I needed were either musty old emporiums that had a slightly creepy vibe, or places that sold supplies for ceremonial magic. Most were at least a day trip out of town, and shipping charges usually curtailed any patronage of those that did take mail orders. I knew that none of the 'niceitieswere necessary, but still, I yearned for the trappings of Witchery.
I had been following a path that incorporated the teachings of Scott Cunningham and used things found in Nature- unpolished stones, mostly- which I would use with organic materials such as flowers, leaves, grasses, sand and wood. It was satisfying and basically no cost. I used white utility candles and tied colored ribbons around them for correspondences,(birthday candles when I could find nothing else, and I still use them today for much of my spell craft) and made my own essential oils and blessing oils out of flowers, fruits and grape seed oil. I anointed candles in olive oil and rolled them in cooking herbs. My magical herbs came from the grocery store. Shells and beach glass came from the nearby seashore, ferns, nettles,thistles and other plants came from the woods, marsh and roadside. My first wand was a small branch from an ancient pear tree that was gnarled and appropriately 'witchy' looking, which I topped with a sliver of quartz I found on a hike in the mountains.
Later, as my practice expanded, I used to make fantastic finds in antique and second hand stores-wooden and pottery bowls, silver spoons, candle holders, antique papers for written spell work, fancy containers to use on my altar, even clothing to re-purpose into ritual garb. Old colored bottles with stoppers became receptacles for herbs, oils and tinctures; some were used as spell bottles. Odd plates became candle holders. I found a beautiful green marble mortar and pestle at the Goodwill for $4.95.
My first 'cauldron' was a round black glazed sugar bowl (half-filled with water for safety sake!), which I used to burn candles in before I purchased a small iron cauldron that I could actually burn things in. The bell on my altar was a chrome plated Christmas 'dinner' bell( I removed the red velvet bow). I made a set of runes out of decorative pebbles, and a pendulum from a small crystal drop on a thin chain (originally a child's necklace). The 'dollar' store was the only place where I could find Chinese almond crescent cookies that I used in ritual; I also bought individual bottles of real apple and grape juice for a fraction of the cost of the grocery store. In one place near the Hispanic part of town, I found the 'mother load'- Florida Water, Money Spray, Bay Rum, small mojo bags, glass charms,figure candles, anointing oils, holy water, saint's candles and packets of exotic 'medicinal' herbs such as rue and hibiscus, and Everclear.
Outside of the old Magickal Childe store in NYC when I was in college, I'd never set foot in a metaphysical/occult/witch shop until many years later. I never had to, because I had learned the cardinal rule of good Witchery...learn to use what you have available and make do with what you have.