It was not the first time that the castle "fell", but it was to be the last, since during the period of Turkish occupation that followed the fall, the Byzantine acropolis was abandoned, while the lower town and the part outside of the walls remained densely populated. The castle was built between 560 and 650, that is, during the reign of Ioustinianos or Heraclius. Since then, many conquerors have passed through, at the end of the 10th century the castle was occupied by the Bulgarian Tsar Samuel, and in 1001 it was recaptured by Basil the Second, the Vulgar Killer. In 1018, Basil the Second partially destroyed the walls so that the castle could not be used as an outpost for the Bulgarian army. In 1204, Servia was occupied by the Franks, and in 1216 it came into the possession of the Epirus governor Theodoros Doukas. Michael the Second Duke, after the battle of Klokotnica (1230), rushed to occupy the castle and immediately repaired the walls. Since 1257, Servia belonged to the Emperor of Nikaia, Theodore the Second Laskaris. In 1341, they passed into the hands of the Serbian Kralli Stefan Dusan, and in 1350 they were recaptured by John the Sixth Kantakouzenos, only to be overrun by the Ottomans for the last time in 1393, who began to build the new settlement outside the castle.

The acropolis of an area of 2.5 hectares, with its stunning view, is now deserted. There are no longer any Byzantine lords to inhabit it. Only its three Christian churches (Agii Theodoroi, Agii Anargyroi and, of course, the Basilica of Katihomenoi), which were not converted into mosques, but were repaired and rebuilt and continued to operate until the 19th century, when the old settlement was finally abandoned.