'Tickit' to a Better CG-CAHPS Survey


The Problem:

San Francisco Department of Public Health needed to survey patients in a timelier, more accessible way

It is important to understand drivers of patient experience to make informed decisions on delivery system redesign, launch better patient centered care initiatives, assist with patient re­tention, and ultimately improve health outcomes. SFDPH's current method of surveying patients about their experience­ - mailed CG-CAHPS surveys- is hampered by delays in report­ing, literacy concerns, and ultimately an inability to use the responses for timely feedback for interventions. 


PDF Download

Presented at: Safety net Conference 2015 San Fransisco


The Solution

Feedback from Patient Advisors at Silver Avenue Family Health Center was Critical to Successful Use of the Technology

SFDPH partnered with Shift Health, a com­pany that designs innovative, easy-to-use graphic based survey tools. TickiT® is an in­teractive mobile questionnaire platform designed to engage patients.

Patient Advisors at SFDPH's pilot site, Silver Avenue Family Health Center, wanted to use technology to better capture the voice of a wider sample of patients, so they worked Closely with the innovation team to tailor the length of the survey and positioning of the tablet based on the needs of the clinic.



Tickit to a better CG-CAHPS Survey test 1.png

Survey Adaptation & Design
• Partnered with start-up company Tickit® Health to create tablet-based survey interface that was visually attractive, 
• icon-based, and simple. 
• Adapted Clinician and Group CAHPS (CG-CAHPS) survey to lower literacy levels of questions and shorten survey length. 

Patient Feedback & Validation
• Elicited feedback from 3 existing patient advisory councils to refine survey questions, interface, and workflow for administration. 
• Conducted 25 in-depth interviews with patients to: Elicit perspectives about care experiences and preferences for reporting feedback to providers/clinics. 
• Validate a tablet-based survey compared to the standard paper-based CG-CAHPS (patients received both surveys in randomized order). 
• Collect feedback about the interface and content of a tablet-based survey to inform future iterations. 

• Analyzed readability and concordance of answers between paper vs. tablet-based questions. Deductive (informed by interview guide) and inductive (open) coding. 

Go with the Patient Flow! 

'Tickit' asset.png

We put the iPad in the exam room, another place where the patient spends "non-value added" time during the visit.

Patients were happy to have the privacy of the room, and were not concerned about missing their provider visit because they were filling it out. as they sometimes were in the waiting room.

The burden of prompting the patient fell on the Medical Assistant (MEA) rather than the front desk staff, and the feedback from MEA staff about the workflow was overwhelmingly positive. When surveyed every day for two weeks, the MEA who had the survey in their exam room reported no interruptions to their normal workflow.


Promising Data from the Pilot Study

finger tablet.png

The SFDPH innovation team tracked the number of patients that required help filling out the survey.

The number was not significantly different in any particular location, and in fact remained quite low throughout - only 18% of responding patients

In August 2015, the care team placed the iPad in an exam room. Though the total number of responses is still lower than in June and May, the average responses/day is greater.

In fact, approximately 4-5 patients pass through an exam room in a given clinic session, so this is promising data for response rate.

What's Next?

Spread of Survey Data Collection to Other Sites

Two more sites will begin collecting surveys using TickiT in October 2015.

The first, Chinatown, is launching in conjunction with their new Patient Advisory Council. The patient advisors at Silver Avenue FHC (pilot site) were crucial in the process of designing the workflow and making the surveys work for both staff and patients, and it was a good way to keep patients engaged with the council work. They hope to accomplish the same at Chinatown.

The second Site launching in October is Tom Waddell Urban Health Center. The Community Advisory Board expressed interest in wanting an easier way to gather feedback from a wider range of patients; TickiT offers a good opportunity for that.

Use of New Patient Experience Feedback

Silver Avenue FHC has settled on a workflow, and the clinic has now moved into the second phase of this innovation: determining how to incorporate real-time patient experience data into its quality improvement work.

In choosing the focus of their quality improvement groups in the coming year, the Quality Improvement Committee at Silver Avenue FHC now can choose from the CG-CAHPS questions that they have been collecting from TickiT.

Another way that Tickit data will be used at SAFHC is through provider-level dashboards. The Tickit data is not broken down by provider, but there will be clinic-wide patient experience data overtime included in the dashboards.

Addition of PDSA-Tailored Questions

Silver Avenue FHC has worked with its Patient Advisory Council to determine the four additional questions that will be added to the end of their shortened CG-CAHPS survey (per the agreement with the vendor).

These questions pertain to PDSAs that the staff has been conducting together with patient advisors in the CoDesign program with the Center for Care Innovations.

For example. one work group is focusing on raising awareness about the SFHN-wide Nurse Advice Line. The question they decided to add to TickiT was "Have you heard of the NAL?" so that over time, they can see whether their efforts to improve awareness are working.


This project was supported by the UCSF CTSI-Strategic Opportunities Support (SOS) Program. Dr. Lyles was supported by a career development award from AHRQ (R00HS022408). 
Drs. Sarkar and Lyles receive support from AHRQ (R24HS022047). 



InnovationHubs, Cahps and the SFDPH