This paper presents UMASS_BioNLP team participation in the MEDIQA-Chat 2023
shared task for Task-A and Task-C. We focus especially on Task-C and propose a
novel LLMs cooperation system named a doctor-patient loop to generate
high-quality conversation data sets. The experiment results demonstrate that
our approaches yield reasonable performance as evaluated by automatic metrics
such as ROUGE, medical concept recall, BLEU, and Self-BLEU. Furthermore, we
conducted a comparative analysis between our proposed method and ChatGPT and
GPT-4. This analysis also investigates the potential of utilizing cooperation
LLMs to generate high-quality datasets