It really depends what we're on about bringing back. If it needs to go in a vessel to come back, that vessel needs to get to the moon in the first place. It's all cost for the sake of overcoming gravity. I still think that what we need to do is process and manufacture in space - find a way of using the characteristics of Space, the Moon, or Mars to the benefit of a process (e.g. could we make some kind of alloy more easily in the vacuum of space, with those ambient temperatures, and abundant solar energy? Could we 3D print that alloy into more complex shapes more easily in a low-G environment?). I think the notion that we could mine the Moon for something like Iron Ore, and bring that back to earth makes little sense.
Oh right. I was talking mainly about raw materials, ore, metals, etc. as you'd said not viable. Stuff where it's just about the Moon having an abundance of material that is less easily accessible or unavailable on Earth. Iron ore is probably always going to be sufficiently available on earth, but depending on the geology of the Moon there may be other things. If the price for a raw material is in the millions of dollars per tonne, you can go a long way to find a rich source.
Bringing ore back to Earth makes perfect sense if the infrastructure is already in place. It's going to be harder to mine on the Moon than on Earth but this is already assuming that whatever ore is available on Earth is financially not viable to mine. And there are still advantages unavailable on Earth, like an electric catapult setup is going to be arguably cheaper to use to distribute your product internationally than standard shipping means - the power is basically free and it costs you the same to land your product anywhere there's a large enough clear area.
One wouldn't have thought it would have been worth mining bird feces and shipping it halfway around the world on a sailboat, but that
sure happened.
An easy imaginary example is the movie
Moon. Helium-3 is required as a fuel and the Moon is the closest place with high enough concentrations to make collection viable, so there's a mining base on the Moon. Obviously that's science fiction, but it's not really that hard to imagine that the Earth becomes scarce enough of some highly useful resource that mining it on the Moon is reasonable. Doubly so if the problems of living and working on the Moon have already been solved through prior research missions to establish habitats and industrial production facilities.
If it's processing and manufacture, then that's different and there would absolutely be restrictions on how much G and the like the items could be subjected to. But for some stuff it's probably still fine, alloys for example. And even more fragile things are still doable, it's just harder. If an escape capsule can get humans (who are not exactly the most durable things ever made) from the ISS down to the ground in one piece, I feel like with a little engineering one could ship most things if you really wanted to. As with many things, it would come down to whether it was economic, not whether it was possible. And without getting into specifics that's a question that's impossible to answer.
If you're into this stuff The Moon is a Harsh Mistress as suggested by
@BobK is a good book. It outlines a Lunar society that is technically self-sufficient undergoing the transition from a colony being bled for raw materials to a state in it's own right trading on fair terms with the nations of Earth. It's not super hard on the science, but there's plenty there about a potential manufacturing Lunar society and how it might function if you're looking for it. As a starting point for a discussion about what Lunar mining and manufacturing might look like, you could do a lot worse.