As many scholars emphasize, including our Stanford guiding principles and Stanford’s Institute for Human-centered AI, advancements in AI need to be centered on improving the human condition. This places importance on augmenting human capabilities as opposed to replacing humans.  

Example metrics for evaluating human-centered creativity:

1. Does the use of AI devalue personal involvement or outcomes in any way? If yes, find an alternative approach to eliminate or mitigate that as much as possible.
2. Are humans being replaced in any capacity? If yes, emphasize the importance of augmenting the role of the human in the creative process that surrounds our applications of AI. 
3. Who has access to the tools? If only a select few, emphasize the need to make these tools available to broad communities.

Ultimately, human-centered creativity values the promotion of justice and individual autonomy and enacts measures to prevent bias and discrimination. It requires establishing governance and evolving guidelines for best practice. 

The notion of purposeful innovation is connected to human-centered creativity and encompasses the exploration and use of AI driven by needs and articulated objectives. This requires up front time developing empathy for  learners and patients to ensure that tools and innovation are being thoughtfully designed so that those needs are addressed as intentional outcomes.

Example metrics for evaluating purposeful innovation:

1. How many perspectives were brought into the product or project design? If only some, explore including more diverse contributors and collaborators.
2. Who is at the center of the innovation? Ensure learners and patients (our end users and most valuable community) are considered at every step.
3. Is there a governance process in place? If not, create a model to evaluate each stage of product or project development to ensure objectives are aligned. 

Bringing in an array of perspectives to inform purpose fosters equity, and developing processes for evaluating objectives and outcomes necessitates the development of governance; both of these qualities are central to the FURM assessment model.

A foundational consideration for AI use across all sectors, including medical education and patient care, is the role of human oversight and its relevant components, including accountability and explicit transparency.  

Example metrics for evaluating ethical standards:

1. Is there a specific line of authority identified within the cycle of AI application? If not, establish lines of authority and governance processes.

2. Is there a clear process for human review, correction, or intervention within the cycle of AI application? If not, work to embed oversight across multiple checkpoints within the AI implementation.

3. Are AI-developed materials reviewed by subject matter experts for accuracy, clarity, and learner engagement? Faculty and instructional designers review for content, tone, and educational intent, with transparent acknowledgment of AI assistance and a log of review and approval.

These attributes encompass applying high levels of rigor and quality to all of our work, abiding by laws and university policies, and considering legal and privacy implications involved in sharing private and sensitive Stanford and third-party data and information with generative AI tools.

Continual refinement and iteration are necessary to ensure responsible integration of AI for medical education.  As technologies and clinical practices advance, our approaches must adapt to new capabilities, ethical considerations, and learner needs. 

Example metrics for evaluating continual refinement and iteration:

1. Do AI materials and applications include continuous evaluation and human oversight? If not, build in iterative feedback loops, performance metrics, and governance structures to inform ongoing improvement.

2. Do AI-made media have regular accessibility audits? Ensure that educational media initially enhanced by AI (like captioning, audio description, or visual layouts) is periodically audited by accessibility experts, and findings are used to retrain AI tools and update future multimedia outputs.

3. Are AI-developed materials reviewed by subject matter experts for accuracy, clarity, and learner engagement? Faculty and instructional designers should review for content, tone, and educational intent, with transparent acknowledgment of AI assistance and a log of review and approval.

Iterative development allows us to evaluate outcomes, incorporate feedback, and align innovations with educational goals, while fostering a culture of curiosity, reflection, and improvement.  It is critical not only for technical excellence but also for maintaining trust, relevance, and integrity in our work.