Wind caves or aeolian caves are formed by the forces of wind, sculpturing the rock with sand like a sandblast. The force of the wind works where the wind blows, and as the interior of the caves tends to be windless, wind caves are generally not very deep. They form overhangs and huge caverns. Very common are natural bridges and short through caves, often of impressive size.
The wind transports solid material, organic remains and sand. This works very well in arid areas, where the sand is not covered by vegetation or agglomerated by water. Additionally the wind speed is not lowered down by the vegetation. The fine sand reaches high speeds and works like a sandblast on the rocks. This process is called wind erosion.
The erosion works everywhere, where the wind has enough force. But edges tend to break the wind, which increases speed by producing eddies. This results in higher erosion at the edges, which results in generally rounded forms. But the resulting surface is also influenced by the resistance of the rock, which depends on its hardness. Softer layers are eroded faster forming the caves, the harder layers form bridges and overhangs.
Unfortunately the term wind cave is also used for something completely different. Caves with heavy wind blowing in or out of the entrance are also called wind caves. Both have the wind in common, but they are not formed by the wind. In contrary, they are generally karst caves, as the physical processes which produce the blowing wind require a large cave system. Those are called Blowholes.