The circle of walls built by Leo IV in the mid-ninth century formed the eastern, bulwark of the popular suburb that grew up between the Tiber and St.Peter's, and that spread out behind the basilica, along the top of the Mons Vaticanus, which formed western and southwestern boundary.
How the hilly area between the apse and the walls, today a garden, was organized in the Middle Ages, is unknown. It certainly was not built upon, and it probably was largely uncultivated. It may have been just this possibility of having a large garden (or more precisely, a protected cultivated area) which induced the Pope to built the residence in the Vatican.
For this purpose he acquired the hill, today occupied by the Cortile di San Damaso and the palace of the Sixtus V, and the entire valley (which was later made over to form the Cortile del Belvedere and Cortile della Pigna) up to the outer slope of Mons Sancti Egidi (the hill atop which Innocent VIII would build the palazzetto del Belvedere at the end of the fifteenth century) extending the area of the Vatican to its natural boundaries on the north and north-west.
To protect his new acquisitions Nicholas built a sturdy circle of walls that joined up with the fortified enceinte of Leo IV. He brought water to the area, arid planted vines and fruit trees on the site of teh present Belvedere and pigna courtyards. The slopes to the west remained wooded.
Nicholas V (1447 - 55) was the first to conceive a series of gardens that would have a practical application in the ceremonies of the papal court and that would also be used for the pope's personal enjoyment. He probably extended their area to the north of the tower that he had built to protect the Porta San Pietro (which for this reason was also called the Porta Viridaria or Garden Gate), along the Via Franchigena , the medieval road connecting Rome with the North.
A contemporary writer describes a system of large and small gardens, fountains, a fish pool, and an enclosure for rabbits. Between the tower and the palace, in the area now occupied by the Cortile di San Damaso and the Palace of Sixtus V, lay the hortus segretus or private garden, the area closest to the palace, which was used for entertaining guests. At the end of fifteenth century a loggia was built to plans by Antonio del Peìlaiolo atop the boundary walls at the north end of the gardens. The loggia, which overlooked the open country outside the walls, was called the Belvedere and was originally conceived as a place where Innocent VIII could stop and rest during his walks. Later two small rooms were added on the east side to allow longer stays.
In the first decade of the sixteenth century, during the papacy of Julius II, Donato Bramante realised a bold, theatrical plan for transforming the valley between the palace and the Belvedere. He made the valley over to form three stepped terraces connected by staircases. (His design would later be broken up—first by the Library of Sixtus V, and then by the Braccio Nuovo or new wing of the Museums - to form the Cortili del Belvedere, della Biblioteca and della Pigna).
In place of Nicholas III's enclosure, Bramante built a great rectilinear defensive wall whose inner façade was elegantly adorned by superimposed blind arcades in correspondence with the covered galleries that linked the wall with the palace, bridging the various levels of the garden. The uppermost gallery (the only one which corresponds to the present Cortile della Pigna) led directly from the papal residence to the Belvedere.
During the sixteenth century the lowest terrace (today the Cortile del Belvedere) was used for plays and receptions, whereas the upper terrace, especially the present day Cortile della Pigna, were kept as gardens and merged with the other gardens that stretched over the site today occupied by the Pinacoteca, the Giardino Quadrato, and the Pio-Clementine Museum.