The church was a secular college on the seaward side of the Cathedral is also known as Kirheugh, St Mary of the Culdees and Church of St Mary of Kilrymont. It was perhaps Scotland’s first collegiate church comprising an association of clergy known as the Culdees or Céli Dé, "vassals of God". The church lasted for several centuries, but did not survive the Scottish Reformation. The ruins are of the monastery where St Rule is likely to have brought the sacred bones of Andrew.

Until the foundation of the Augustine priory in 1140, the Céli Dé and the seven clerics known as the personae (parsons) are the only known clerics of the cathedral. The original church was located at Kirkheugh or Kirkhill. This is where the modern North Street and South Street converge. Burials point to the original sixth century religious site. Possibly pre-dating the alleged eighth-century century foundation of the monastery, and point to a small religious community from the sixth-century.[ The church may have been pulled down by reforming Protestants as late as 1159. The remaining foundations of the church show that it was cross-shaped, and possessed no aisle. It is a much overlooked part of a very historic Cathedral town.