Building an AI Doctor

As co-founder and head of product and design at Kips Health, I led the vision and product development of our flagship product, the AI Patient Messenger. Our goal was to offload patient communications completely to an LLM, allowing care providers more time to do what matters most - actually providing care. This is my postmortem to document some of the challenges we faced, why we made specific design decisions, and little nuggets for what I think the future of AI X Health could look like.

So, why AI messaging?

We spent over a month talking to hundreds of medical professionals about their hardest problems. While most medical niches are overworked, physical therapists and musculoskeletal (MSK) specialists experienced some of the highest degrees of burnout. Over 50% of these providers experience it.

Then, in 2023, a newly passed billing code added even more burnout fuel to MSK providers. This new code reimbursed MSK specialists for communicating with their patients outside of the clinic, like calling, messaging, or texting. All of a sudden, clinic owners put even more pressure on providers to talk to their patients - even when they were off hours or outside the clinic.

Now imagine you're that burned out doctor. We give you a bot that handles communication, facading as you. Suddenly, you can check in with patients, answer their questions, and collect reimbursements - all while chilling on your couch (or whatever doctors do in the tiny number of minutes they can rest).

Selling a dream

Despite our product being the only RTM app (as far as I know of) that literally required 0 seconds of time to send messages, clinic owners still asked us, "I'm really busy, will I have to spend any time on this?"

The bar for RTM software as it stands is still very high. So high that even full AI messaging might not be worth the documentation, setup, and training time for clinics. I do believe one day there will be a true AI doctor in our pockets though. One that knows us so personally (through tracking our wearable data or knowing our health history) that it can proactively consult with us about our health, make recommendations, and diagnoses. Accurately too!

"Can I really trust your AI to answer as me?"

Was probably the most popular concern we got during sales demos - and for good reason. If the AI decides to tell someone that just sprained an ankle that they have cancer, we're all collectively screwed.

We addressed this issue by using another LLM to grade every AI reply that would be sent. It would then deterministically decide if the message is a go or no go. This grading strategy is still pretty popular in the LLM evaluation space. While not 100% safe, it assuaged the concerns of most customers.

Integrating with existing dinosaur tech

As a 2-person startup (most of the time), you simply don't have the bandwidth to engineer a full integration with some obscure health record system for one customer. Unfortunately, we found that not being able to do that is one of the biggest dealbreakers for potential customers in healthcare.

But that's why you keep searching for that early customer who will use your product despite that dealbreaker. On side note, a universal healthcare integration API is probably one of the biggest opportunities for a startup in this space. So yeah, someone should do it.

So easy your grandma can use it

Probably a great usability benchmark to hit. Can your grandma, who doesn't know how to turn on a TV, still use your product with ease? We made it easy. We really tried to.

Instead of setting up a mobile app because we were tEcH STaRTuP FOUnDeRs, we just sent texts directly to patients. Least amount of friction and something they already use. Except even the habit of texting is not well-ingrained among Medicare age users.

If I were to build this again, I would make it voice-based. 80% of Americans over 65 still prefer calls to texts. Not everyone is as chronically online as Gen Z or alpha.