Baptisms are important events. In Acts 2:38, the Apostle Peter reminds people that they must repent and be baptized in the name of Jesus for the remission of sins and to receive the gift of the Holy Spirit.
When the Holy Spirit descended upon Jesus in the river Jordan as Jesus was being baptized by John the Baptism, the Holy Spirit came in the form of a dove.
It is therefore fitting to commemorate baptisms with a release of doves, as they serve as a reminder of the free flight of the Holy Spirit and its gifts.