These caves were carved out of solid sandstone on a hillside in the 4th to 5th centuries A.D. There are several caves and the best known largest one has four stories with a huge recreated statue of Vishnu in a reclining posture, sculpted from a single block of granite inside the second floor. Undavalli caves are an example of how many Buddhist artifacts and stupas in Andhra were converted into Hindu temples and deities. It was originally a Jain cave resembling the architecture of Udayagiri and Khandgiri. The main cave is one of the earliest examples of Gupta architecture, primarily primitive rock-cut monastery cells carved into the sandstone hills. Initially the caves were shaped as a Jain abode and the first floor abode still retains the Jain style; the vihara exhibits Jain monastics and includes tirthankara sculptures. This first level of the cave is a carved vihara and includes Buddhist art work. The site served as the Bhikkhu monastic complex during ancient period. The walls of the caves display sculptures carved by skilled craftsmen.
The caves are surrounded by the green countryside. From the high hill above the cave overlooking the Krishna River many fine specimens of rock cut architecture can be seen.