How the brain enables individuals to adapt behavior to their partner is key to understanding social exchange. For example, courtship behavior involves sensorimotor processing of signals by prospective partners that can result in behavioral dialogue, such as stereotyped movements and singing. The courtship behavior of Drosophila melanogaster males with their partners, which are usually female but can also be male, involves singing. To investigate how behavioral feedback and sensorimotor processing contribute to flexible social interactions, we compared the courtship behavior and singing of male D. melanogaster towards males and females. Quantitative analysis of their interactions revealed that while underlying courtship and song rules are unaffected by the sex of the partner, the behavioral dynamics and song sequences differ by partner sex. This divergence stems from sex-specific behavioral feedback: females decelerate to song, while males orient towards the singer. Moreover, optogenetic manipulations reveal that the partners responses are driven by a specific neural circuit that links song detection with arousal and social decisions. Our findings demonstrate that flexible social behaviors can arise from fixed sensorimotor rules through a context-dependent selection facilitated by the partners behavioral feedback. More broadly, our results reveal compositionality as a key mechanism for achieving behavioral flexibility during complex social interactions such as courtship.