Artificial Intelligence (AI), over the past decade, has improved substantially in its ability to mimic regular conversation and provide more accurate responses (Yin et al., 2021). AI applications, such as ChatGPT, have recently developed the capacity to reject requests that are inappropriate or beyond the application’s reach, challenge incorrect assumptions, and admit mistakes (Makridakis et al., 2023). Freyer and colleagues’ (2019) even found that college students interacting with Chatbots reported greater curiosity and perseverance than those interacting similarly with peers. Despite these advancements, there are legitimate concerns over the use of AI, especially within education. Responses from AI apps continue to be unnatural, misleading, inaccurate, vague, and biased (Fryer et al., 2019; Sallam et al., 2023). Research shows a decline in the quality of information if educators become over-reliant on AI (Chiu et al., 2023a). Still, the capacity for AI to assist educators and motivate students has been displayed (Chen et al., 2020; Laupichler et al., 2022; Ng et al., 2023a, 2023b; Salas-Pilco et al., 2022; Sanusi et al., 2022; Su et al., 2022; Tan et al., 2022; Wangenheim et al., 2021).

This presentation discusses a mixed methods study on a randomized control trial evaluating AI safeguards. Possible solutions from this study include: (a) providing engineered detailed prompts within AI, (b) pulling data from a knowledgeable database by trusted experts, (c) providing specific questions within the prompt of response type warranted, (d) creating a clear pedagogical focus, and (e) using AI as one source within many sources to gather outside perspectives rather than replace human data collection. It will also discuss prompting questions to improve AI accuracy, such as: “Was the data the AI was trained on representative of the population affected by subsequent decisions?” and “Is the information given well-balanced and derived from an analysis of information from trustworthy sources?”