Expert Reactions to the New FDA Guidance (July 2024)
by Katie Curtis
Knowledge Tasks: Method of Evaluation or Type of Task?
The U.S. Food and Drug Administration (FDA) released a new draft guidance document in July 2024 entitled, “Purpose and Content of Use-Related Risk Analyses for Drugs, Biological Products, and Combination Products” (see our previous blog post for an overview of the new draft guidance). One notable aspect of the FDA URRA guidance is a new term in the Glossary: knowledge task.
Redefining Knowledge Tasks
Historically, knowledge task has been used to describe a method of evaluation employed during HF validation studies. Knowledge tasks are designed to assess important aspects of use that cannot be evaluated through simulated use testing because they involve users’ understanding of information, which is difficult to ascertain by observing user behavior. Knowledge tasks are assessed through direct questioning of participants using open-ended, naturally worded questions.1
The 2024 draft guidance proposes a different definition for knowledge task. Rather than describing a method of evaluation, the guidance proposes that a knowledge task is a type of task associated with a product.
Previous FDA guidance documents have referred to all tasks associated with use of a product using the general term task 1,2. However, the URRA guidance draws a new distinction between two different types of tasks: user tasks and knowledge tasks. User tasks in this draft guidance are defined as, "an action or set of actions performed by a user to achieve a specific goal”, which is the definition previously used for the broader category of tasks, while knowledge tasks pertain to assessing information in the labeling.
Knowledge tasks are defined as, “tasks that require user understanding of information provided to the user in the product’s labeling and that are not typically or easily evaluated through observation of simulated use. Rather, knowledge tasks are generally evaluated through knowledge-based questions.” Note that the evaluation method previously referred to as knowledge tasks is referred to as “knowledge-based questions”, differentiating the method of evaluation from the task type.
User vs. Knowledge Tasks – Where’s the Line?
While the URRA guidance seeks to provide guidance on identifying tasks associated with use of a product, we look forward to additional clarification from the Agency regarding this new distinction between user tasks and knowledge tasks. Here are some of Design Science’s initial thoughts:
First, the definition of knowledge tasks is limited to comprehension of information in the labeling. However, knowledge can be acquired through a product’s user interface (UI) more broadly – not just through labeling. For example, consider an on-body injector (OBI) with a status light. The status light informs users of the status of the injection, imparting knowledge through a physical component of the UI. Based on the definitions in the guidance, a user’s interpretation of the OBI status light does not fit neatly under the definition of a user task, because it does not involve physical interaction with the product, or a knowledge task, because it does not pertain to the labeling. How should such a task be defined?
Second, this example illustrates an issue with the newly proposed paradigm of differentiating physical versus knowledge tasks. It is difficult to make this distinction at the task level prior to completing other steps in the URRA process, such as identifying potential use errors and risk controls. While feasible during the iterative process of risk management, we are unsure if the categorization of URRA tasks as user tasks versus knowledge tasks is of significance. The important question is how each task will be evaluated in an HF validation study (assuming validation is deemed necessary), which does not neatly line up with the definition of tasks in this draft guidance. The risk control measures associated with each task/use error will dictate the appropriate method of evaluation, which may include simulated use testing and knowledge-based questions, as shown in the OBI example.
Interpreting Ambiguity in the Definition
HF practitioners may encounter another challenge when interpreting the new knowledge task definition. The definition includes the verbiage, “not typically or easily evaluated through observation of simulated use”. The terms “typically” and “easily” are subjective, which creates ambiguity in the definition. Based on our experience interacting with the FDA, we recommend interpreting this definition conservatively, as the Agency has a strong preference for assessing tasks through simulated use over knowledge assessment whenever possible for combination products.
For example, some may consider it “difficult” to directly observe performance on tasks such as checking the expiration date or medication name on a product. However, recent FDA feedback shows that the Agency believes such tasks can and should be evaluated through simulated use. In such cases, we use a combined approach of observation coupled with follow up questions during the root cause interview to accurately assess task performance that is not “easily” observable, when necessary.
Share your Comments
Design Science is excited to contribute to the comments being solicited by the FDA on the new draft URRA guidance, and we look forward to seeing additional clarification regarding the topic of knowledge tasks in the finalized version of the guidance.
What are your thoughts on the new definition of knowledge tasks? Let us know on LinkedIn and be sure to submit formal feedback to the FDA by September 9, 2024 (Federal Register Docket No. FDA-2024-D-2484).
1 FDA Final Guidance for Industry and FDA Staff: Applying Human Factors and Usability Engineering to Medical Devices (2016)
2 FDA Draft Guidance for Industry and FDA Staff: Contents of a Complete Submission for Threshold Analyses and Human Factors Submissions to Drug and Biologic Applications (2018)
Share this entry