Gypsum is not too frequent, and it is soluble to water, so weathering is very fast. Still, there are some karst areas in gypsum on the World, with caves and other karst features.
Gypsum caves are karst caves. They are formed by water weathering the gypsum.
Gypsum is also a valuable good, used e.g. for building houses, so there are some mines in Gypsum. But sometimes the gypsum was just an easy way to get to the valuables below.
Gypsum is a very soft stone, so it is easy to build big chambers inside this rock.