An Underwater Cave is completely submerged, it is filled with water and the only way to explore it is by diving.
Karst has caves, and those holes in the rock are interconnected and filled with water, which is called an aquifer. Above the water table of this aquifer, the caves do not have water, they are called dry or fossil, the scientific term is vadose cave. Below the water table they are completely filled by water and are called underwater cave, the scientific term is phreatic cave. Then there is the area around the water table, caves in this area are typically filled with air but have a river flowing through the cave. Those caves are called river caves, the scientific term is epiphreatic cave.
Underwater Caves are located below the water table, the aquifer moves through cracks which are slowly but continually widened by dissolution. This is typically the first stage in cave development. While the floods run through the river cave level above, this lower level is for the normal water flow. This structure is also visible on the surface, typical karst springs have a lower perennial spring which is connected to the underwater cave, and a higher intermittent spring which is connected to the river cave.
However, there is a second type of underwater cave, which contains speleothems. Speleothems form in dry or fossil cave, so those caves were once for some time in the vadose zone, but are now again in the phreatic zone. It’s unlikely the cave moved, so the obvious conclusion is that the water level changed dramatically. It went up and the cave was flooded. During the last glacial or cold age, for several ten thousand years, the land was covered by glaciers, Scandinavia had an ice shield which was up to 3.5 km thick. As a result, the sea was around 100 m deeper than today, in the peak even more. The Chanel between Great Britain and France was dry land, the coastline was much further out. And the caves in the 100 m zone became fossil and speleothems grew. When the cold age ended, about 12,000 years ago, the sea level rose over several thousand years to its current level. The caves were flooded, some even have seawater or brackish water inside.
If you guess that this is a rare exception, you are completely wrong, actually the largest underwater caves we know today are of this type. The water-filled caves of the Yucatan peninsula in Mexico have this type. The Sistema Ox Bel Ha is currently the second longest known cave system in the world with a length of 308 km [2024], the Sistema Sac Actun is the fourth longest known cave system in the world with a length of 234 km [2024]. Both are underwater cave systems located in the Yucatan peninsula.