Work began on the house in 1831 to a design by the architect John Verge and was finished about 1835. Verge's design for the house was based on the Palladian principle of a central two storied block flanked by symmetrical pavilions; the pavilion to the north west extended into an enclosed courtyard. The house was built of stuccoed sandstock brick on sandstone foundations, which doubled as a cellar. Window mouldings, porticos, parapets and the single-piece columns were made of the local Hawkesbury sandstone. The labour was provided by convicts assigned to the Macarthurs.
Verge's design of the house, although based on a European pattern, has proved appropriate for the extremes of the Australian climate and to the changing patterns of social behaviour. The only major addition occurred in the 1880s, when a second storey was added to the North West wing. The informal layout of the rooms contrasts with the formal exterior and accurately reflects fashions of the 1830s.