Iceland is a volcanic island, formed by eruptions caused by the sea floor spreading in the middle of the Atlantic Ocean. It sits on the convergent rim, with the plate boundary running right through the island. The plates move from here to the east and to the west, continually increasing the (geographic) distance between Europe and America. The space is filled by basaltic lava coming up from the Earth's mantle. As a result, there are volcanic rocks like basalt and tufa, and sedimentary rocks, formed by erosion of volcanic rocks. Especially the large glaciers of the island produce a decent amount of moraine. However, there are no soluble rocks like limestone or gypsum, which are formed as biogen sedimentary rocks. There are no reefs so far to the north, which would produce reef limestone.
As a result Iceland has generally three types of caves,
Iceland has an enormous number of lava tubes. They are located in the holocene lava flows, formed during the last 12,000 years, that cover about 10% of Iceland. Older lava tubes may have existed, but they are already destroyed by erosion.
Much rarer are all other types of lava caves. Blister caves, which formed by gas bubbles in the lava, are obviously just a single ellopsoid cavern. We do not have any information about blister caves on iceland, which are open for visitors, but they may sometimes be found on trekking tours, as they make small but good shelters.
The large Icelandic lava caves can be visited with a minimum amount of gear. A pair of good hiking boots, a helmet, thick gloves (rubber, leather) and a good light source are necessary. However, visitors should be very careful and tell someone where they are going. Although the caves are easy in comparision to other wild caves, they are still wild caves. The following caves may be visited on guided cave trekking tours, which is much better than going on your own.
A glacier cave is not an ice cave. A glacier cave is formed inside a glacier, inside the ice, an ice cave on the other hand contains ice, but its walls are rock. A special feature of Iceland are glacier caves formed by thermal springs. The glacier caves are not very well known, exploration started only a few years ago.
Glacier caves are very dangerous, they should only be visited by highly skilled cavers! Visits require special ice climbing gear.
Many volcanic rocks, like tufa, contain a high amount of gas. They are soft and easily eroded, they have cracks caused by the cooling, which are now used by water to destroy the rock from inside. Water is available in high amounts, high precipitation and melting water from the glaciers forms wild, energy rich rivers. The low temperatures destroy rocks too, as frost freezes the water in the cracks of the rock, the water expands and the rocks almost explode by the forces from inside.
The erosion of the rivers causes the formation of river caves, both in lava or tufa and in clastic sediments like moraine material. Some of those caves are rather huge, but most of them are only shelters in rather soft rock. They are not very interesting from the speleological view, and often dangerous.