Ants are on peony to get the sugary exudate from the flower buds. They are not necessary for proper blooming and do not spread botrytis blight.
Japanese beetles feed on the late blooms or the leaves. Applications of Sevin, rotenone, or methoxychlor control these pests.
Rose chafer is often mistaken for japanese beetle. These light tan beetles with spindly legs eat the flowers. Rose chafers are killed by sprays of Sevin, rotenone, or methoxychlor but a reduction in the population may not be noticeable.