In this report, we describe an experiment in which the effect of the presence (+G) and absence of gestures (-G) in an automatically generated dialogue between two embodied conversational agents was examined. In particular, we focused on correlations between +/-G and 1. the ability of subjects who witnessed the dialogue to correctly answer questions regarding the content of the dialogue and 2. judgments on how engaging a dialogue was. We also tested whether subjects would judge the politeness of the characters in line with the system settings for this parameter. Finally, we asked our subjects for any comments. This yielded some interesting results regarding the relation between +/-G and judgments about the quality of speech synthesis.