Both in Late Antiquity and in Byzantine times, houses were built within the walls of the city or castle, the greater part of which was given over to residential areas. Although houses in early Byzantium were arranged on blocks on either side of main or secondary streets, in the middle and late periods they either stood right next to each other, with a dividing wall in between, or were scattered according to the lie of the land.

In large imperial cities, ground floor houses arranged around a central courtyard could be found next to two-storey or multi-level buildings divided into apartments. For house building the Byzantines usually used stones, brick and wood with sand and lime mortar, or mud for humbler buildings. The low grade building materials in use throughout Byzantine times may account for the fact that very few examples of secular architecture have survived to the present time.
 
The legal provisions that regulated town planning and building are indicative of the special interest taken in aesthetics and hygiene. There were regulations on the number of storeys a house could have; on balcony and staircase construction; and on the distance between houses. In some areas such as Constantinople, there was even a law against obstructing the view out to sea. Wealthy owners in particular were obliged to take care of their house fronts, so as to adorn the city and give pleasure to passers by.

One distinguishing feature of Middle Byzantine houses was the use of their ground floors as storage areas, with installations such as large jars and stone-built silos, which would have looked somewhat out of place in the urban environment. The well-to-do kept their houses in the cities, though we are unsure whether they lived in distinct neighborhoods or right next to poorer dwellings. Mystras and Adrianople are illustrative of the former tendency, as the mansions of the wealthy are concentrated in the upper section of both towns. In several cities the Jewish population may similarly have lived in a separate district, though this discrimination seems to have waned over the years; the same might also have been true of foreign merchants, mainly from Italy, who appear to have lived outside the walls or in separate communities.