The castle was built on a peninsula in the small bay of Porto Palermo, known in antiquity as the Bay of Panormos, on the road connecting Himara with Qeparo. The castle was constructed by Ali Pasha Tepelena on the ruins of an earlier Venetian castle. It has a triangular plan with three strong bastions at its corners. Its dimensions are 150 m x 400 m, and the height of the walls reaches 20 m. The entrance is located in the middle of the southern wall. On the seaside, between two bastions, a trapezoidal area is surrounded by walls equipped with gun ports. The rest of the castle consists of a large number of rooms covered with vaults, ending in an upper terrace. Ali Pasha primarily used the castle as a military post but also as a prison, a function that was also used during the communist regime for political prisoners and interned families during the dictatorship.