AstraZeneca: Using AI to Scale Cancer Detection SaMD

Stage 1: Usability & Opportunity (2 Weeks)

Task 1: Initial Usability Issues & Scope - The goal for this product team was to encourage increased usage of the SaMD to bring more professionals into the AstraZeneca HealthTech ecosystem. With this solution being a recent acquisition from a start-up, the prior focus had been purely on technical considerations, with no attention paid to the user experience.

Starting with the deployed MVP, my first goal was to quickly identify and provide feasible solutions to solve for the usability limitations.

Quickly Understanding Opportunities & Costs: Neilson Norman Heuristic Analysis

  • For a small product team with limited resources, the central challenge was to improve usability while getting the most valuable possible.
  • Undertaking a complete audit of the solution, I organized usability principles clearly with needed detail, categorized issues by the type of work needed, and prioritized them to make the most of resources.
  • Delivered ahead of schedule, this provided some maneuvering room for rapid user research to add contextual depth to these issue.

Task 2: Exploring New Opportunities - During the process of determining usability improvements, users shared several instances of their workflow being bogged down by laborious, repetitive, and focus-heavy tasks.

Sensing a potential opportunity to build more value on the initial project goal (i.e. providing a more usable experience), I spent the time available from the original project scope to dive deeper into how current processes worked and ideated potential improvements.

Establishing Deeper Context & Understanding: User Interviews & Workflow Analysis

  • As part of the initial usability effort, I expanded the scope of scheduled interviews to learn more about the users’ tasks and prioritie.
  • I was surprised by how labor-intensive the process was, and realized that usability improvements were only the first step.

Bringing More Value to the Table: Concept Development & Pitch

  • To develop a feasible concept, I mapped out the existing workflows in detail to account for as many factors as possible.
  • I realized that current AI models were extremely well suited for the core task of examining large image files for well-understood indicators.
  • Consulting with the product owner and my department head, I developed a short pitch presentation for C-Suite decision-makers that outlined potential benefits, costs, and needed resources.

Stage 2: AI Augmentation (6 Weeks)

Task 1: Training & Refining the AI Model - Before any other consideration, the accuracy of the model needed to be established and rigorously tested before being deployed. This goal required a series of critical systems-thinking design decisions to keep any form of harm from users or the patients that depended on them.

AI Accuracy via Focused Training: HIPPA-Compliant Dataset Design

  • With the realization that more broadly trained AI models are more prone to mistakes, I worked with SMEs to outline a dataset streamlining plan.
  • This focused training data removed any confounding factors for results generation and increased accuracy, while keeping us HIPAA compliant by removing information that could identify individual.
  • This provided a high initial standard for accuracy, but we quickly realized that additional system design choices could further refine results.

Ending AI Hallucinations via Systems-Thinking: “Escape Hatches” & Confusion Matrix

  • With a high degree of preliminary accuracy, I had to adapt my thinking to include additional system-design factors to eliminate remaining doubts.
  • I was surprised to learn that a simple “escape hatch” option would virtually eliminate the possibility of harmful “false negative” results.
  • I was most proud of the trust this “Human-In-The-Loop” AI solution produced, leading the streamlined workflows and increased adoption.

Task 2: Integrating AI into Existing User Workflows - With an AI model that produced trustworthy results in a fraction of the time, I needed to consider how that functionality would be obvious to users so they could interact with, edit, and have the final say in what the system produced.

Armed with my previously performed usability review (i.e. the original scope of this project), I took advantage of this “two birds, one stone” opportunity to update the interface as needed.

Making AI a Collaborative Partner: Revised Interface Design

  • With this novel functionality, the initial challenge was to clearly communicate its inclusion and capabilities into the existing design framework.
  • Users needed to be aware of both the AI functionality, any results generated by it, and how they could leverage/modify those results.
  • My biggest takeaway from initial usability-testing was that a sense of trust was essential; specifically in how the AI accelerated workflows while maintaining industry-standard results.

© 2016 David Service

Version 3.2.0