350 lines
9.9 KiB
Markdown
350 lines
9.9 KiB
Markdown
# AI Patient Onboarding — Frontend Integration API Reference Guide
|
|
|
|
Base URL: `http://127.0.0.1:8000`
|
|
Protocol: `HTTP / REST` + `JSON` + `multipart/form-data`
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
The AI Patient Onboarding system provides two modes of integration for frontend web/mobile applications:
|
|
|
|
1. **Voice AI Onboarding API** (`/v1/voice-onboarding/...`): Supports voice microphone recording, ASR (Speech-to-Text), synthesized audio response streaming (TTS), Google Gemini AI triage, response timing, and EHR JSON export.
|
|
2. **Text AI Onboarding API** (`/v1/onboarding/...`): Supports pure text chat-based intake, Google Gemini AI reasoning, state extraction, and EHR JSON export.
|
|
|
|
---
|
|
|
|
# SECTION 1: Voice AI Onboarding API (`/v1/voice-onboarding`)
|
|
|
|
### 1.1 Start Voice Onboarding Session
|
|
Initializes a voice onboarding session and returns localized greeting text + synthesized spoken audio URL.
|
|
|
|
- **Endpoint**: `POST /v1/voice-onboarding/start`
|
|
- **Content-Type**: `application/json`
|
|
|
|
#### Request Payload
|
|
```json
|
|
{
|
|
"patient": {
|
|
"id": "PAT-9624",
|
|
"name": "Gokul Varthan",
|
|
"age": 25,
|
|
"gender": "male"
|
|
},
|
|
"category": {
|
|
"type": "general",
|
|
"code": "general_medicine",
|
|
"name": "General Medicine / Primary Care"
|
|
},
|
|
"language": "ta-IN"
|
|
}
|
|
```
|
|
|
|
*Note: `category` is **optional**. If omitted, it defaults to `general_medicine` (Auto-Triage).*
|
|
|
|
#### Supported Languages
|
|
- `ta-IN`: Tamil (Pechu Tamil Spoken Dialect)
|
|
- `hi-IN`: Hindi (Conversational Spoken Hindi)
|
|
- `te-IN`: Telugu (Conversational Telugu)
|
|
- `kn-IN`: Kannada (Conversational Kannada)
|
|
- `en-IN`: Indian English (Empathetic Spoken English)
|
|
|
|
#### Response (`200 OK`)
|
|
```json
|
|
{
|
|
"success": true,
|
|
"session_id": "VOICE-4C5F63F0",
|
|
"patient_id": "PAT-9624",
|
|
"category": {
|
|
"type": "general",
|
|
"code": "general_medicine",
|
|
"name": "General Medicine / Primary Care"
|
|
},
|
|
"language": "ta-IN",
|
|
"status": "in_progress",
|
|
"stage": "greeting",
|
|
"greeting_text": "வணக்கம் Gokul Varthan, நான் உங்க AI அசிஸ்டன்ட் நிலா. உங்களுக்கு உடம்புக்கு என்ன பண்ணுது, என்ன பிரச்சனைன்னு சொல்லுங்க?",
|
|
"audio_url": "/v1/voice-onboarding/audio/aud_32dd00c39aa1",
|
|
"response_time_ms": 36.33,
|
|
"response_time_sec": 0.036
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 1.2 Submit Voice/Text Conversation Turn
|
|
Submits the patient's voice audio recording or text fallback. The backend runs ASR (Sarvam Speech-to-Text), Google Gemini AI reasoning, and returns spoken audio URL + dynamic triage category.
|
|
|
|
- **Endpoint**: `POST /v1/voice-onboarding/{session_id}/turn`
|
|
- **Content-Type**: `multipart/form-data`
|
|
|
|
#### Form Fields
|
|
| Field Name | Type | Required | Description |
|
|
| :--- | :--- | :--- | :--- |
|
|
| `audio` | `File` (WebM/WAV) | Optional | Audio blob recorded by microphone. |
|
|
| `text_fallback` | `string` | Optional | Text entered by patient. **Pass `null` or omit when sending `audio` so Sarvam ASR runs.** |
|
|
|
|
#### React / JavaScript Integration Example
|
|
```javascript
|
|
const sendVoiceTurn = async (sessionId, audioBlob, textInput) => {
|
|
const formData = new FormData();
|
|
if (audioBlob) {
|
|
formData.append('audio', audioBlob, 'recording.webm');
|
|
}
|
|
if (textInput && textInput.trim()) {
|
|
formData.append('text_fallback', textInput.trim());
|
|
}
|
|
|
|
const response = await fetch(`http://127.0.0.1:8000/v1/voice-onboarding/${sessionId}/turn`, {
|
|
method: 'POST',
|
|
body: formData,
|
|
});
|
|
|
|
return await response.json();
|
|
};
|
|
```
|
|
|
|
#### Response (`200 OK`)
|
|
```json
|
|
{
|
|
"success": true,
|
|
"session_id": "VOICE-4C5F63F0",
|
|
"patient_transcript": "எனக்கு ரொம்ப மன அழுத்தம் மற்றும் மனச்சோர்வு இருக்கு",
|
|
"ai_text": "புரியுதுங்க கோகுல், இந்த மன அழுத்தமும் மனச்சோர்வும் எத்தனை நாளா இருக்குன்னு சொல்ல முடியுமா?",
|
|
"stage": "reason_for_visit",
|
|
"completed": false,
|
|
"audio_url": "/v1/voice-onboarding/audio/aud_a1b2c3d4e5f6",
|
|
"suggested_specialty": {
|
|
"type": "speciality",
|
|
"code": "psychiatry",
|
|
"name": "Psychiatry & Mental Health Care",
|
|
"confidence": 0.95,
|
|
"reason": "Automatically classified based on clinical keyword: 'மன அழுத்தம்'"
|
|
},
|
|
"response_time_ms": 1420.50,
|
|
"response_time_sec": 1.421
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 1.3 Stream Spoken Audio (TTS)
|
|
Streams synthesized audio response files.
|
|
|
|
- **Endpoint**: `GET /v1/voice-onboarding/audio/{audio_id}`
|
|
- **Response Headers**: `Content-Type: audio/wav`, `Content-Disposition: inline`
|
|
|
|
#### Audio Playback Code
|
|
```javascript
|
|
const playAudioStream = async (audioUrl) => {
|
|
if (!audioUrl) return;
|
|
const fullUrl = audioUrl.startsWith('http') ? audioUrl : `http://127.0.0.1:8000${audioUrl}`;
|
|
const audio = new Audio(fullUrl);
|
|
try {
|
|
await audio.play();
|
|
} catch (err) {
|
|
console.warn("Audio autoplay blocked by browser:", err);
|
|
}
|
|
};
|
|
```
|
|
|
|
---
|
|
|
|
### 1.4 Complete Voice Session & Export EHR JSON
|
|
Finalizes intake and exports the complete structured EHR JSON.
|
|
|
|
- **Endpoint**: `POST /v1/voice-onboarding/{session_id}/complete`
|
|
- **Content-Type**: `application/json`
|
|
|
|
#### Request Payload
|
|
```json
|
|
{
|
|
"confirmed": true
|
|
}
|
|
```
|
|
|
|
#### Response (`200 OK`)
|
|
```json
|
|
{
|
|
"success": true,
|
|
"session_id": "VOICE-4C5F63F0",
|
|
"status": "submitted_and_cleaned",
|
|
"message": "Voice onboarding successfully saved to patient record.",
|
|
"result": {
|
|
"patient_id": "PAT-9624",
|
|
"category": {
|
|
"type": "speciality",
|
|
"code": "psychiatry",
|
|
"name": "Psychiatry & Mental Health Care"
|
|
},
|
|
"chief_complaint": "Severe mental depression and stress",
|
|
"symptoms": [
|
|
{
|
|
"name": "Mental depression",
|
|
"duration": "2 months",
|
|
"severity": "Moderate",
|
|
"description": "Patient reports persistent sadness and low mood."
|
|
}
|
|
],
|
|
"medical_history": [],
|
|
"medications": [],
|
|
"allergies": ["No Known Drug Allergies (NKDA)"],
|
|
"additional_information": "Referred to specialist psychiatrist.",
|
|
"language": "ta-IN",
|
|
"onboarding_completed": true
|
|
},
|
|
"response_time_ms": 2840.10,
|
|
"response_time_sec": 2.840
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
# SECTION 2: Text AI Onboarding API (`/v1/onboarding`)
|
|
|
|
### 2.1 Start Text Onboarding Session
|
|
- **Endpoint**: `POST /v1/onboarding/start`
|
|
- **Content-Type**: `application/json`
|
|
|
|
#### Request Payload
|
|
```json
|
|
{
|
|
"patient": {
|
|
"id": "PAT-9624",
|
|
"name": "Gokul Varthan",
|
|
"age": 25,
|
|
"gender": "male"
|
|
},
|
|
"category": {
|
|
"type": "general",
|
|
"code": "general_medicine",
|
|
"name": "General Medicine / Primary Care"
|
|
},
|
|
"language": "en"
|
|
}
|
|
```
|
|
|
|
#### Response (`200 OK`)
|
|
```json
|
|
{
|
|
"success": true,
|
|
"session_id": "ONB-8F3A2B1C",
|
|
"patient_id": "PAT-9624",
|
|
"category": {
|
|
"type": "general",
|
|
"code": "general_medicine",
|
|
"name": "General Medicine / Primary Care"
|
|
},
|
|
"language": "en",
|
|
"status": "in_progress",
|
|
"stage": "greeting",
|
|
"message": "Hello Gokul Varthan, welcome to CureSelect Healthcare. Could you please share what health symptoms you are experiencing today?"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 2.2 Send Text Message
|
|
- **Endpoint**: `POST /v1/onboarding/{session_id}/message`
|
|
- **Content-Type**: `application/json`
|
|
|
|
#### Request Payload
|
|
```json
|
|
{
|
|
"message": "I have been experiencing a severe headache and fever since yesterday."
|
|
}
|
|
```
|
|
|
|
#### Response (`200 OK`)
|
|
```json
|
|
{
|
|
"success": true,
|
|
"session_id": "ONB-8F3A2B1C",
|
|
"stage": "symptoms",
|
|
"message": "Understood Gokul. Have you taken any fever or pain reliever medication yet?",
|
|
"status": "in_progress"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 2.3 Complete Text Onboarding & Export EHR JSON
|
|
- **Endpoint**: `POST /v1/onboarding/{session_id}/complete`
|
|
- **Content-Type**: `application/json`
|
|
|
|
#### Request Payload
|
|
```json
|
|
{
|
|
"confirmed": true
|
|
}
|
|
```
|
|
|
|
#### Response (`200 OK`)
|
|
```json
|
|
{
|
|
"success": true,
|
|
"session_id": "ONB-8F3A2B1C",
|
|
"status": "submitted_and_cleaned",
|
|
"message": "Onboarding successfully completed, saved to patient record, and temporary session cleaned.",
|
|
"result": {
|
|
"patient_id": "PAT-9624",
|
|
"category": {
|
|
"type": "general",
|
|
"code": "general_medicine",
|
|
"name": "General Medicine / Primary Care"
|
|
},
|
|
"chief_complaint": "Severe headache and fever",
|
|
"symptoms": [
|
|
{
|
|
"name": "Headache and fever",
|
|
"duration": "1 day",
|
|
"severity": "Severe",
|
|
"description": "Patient reports onset yesterday."
|
|
}
|
|
],
|
|
"medical_history": [],
|
|
"medications": [],
|
|
"allergies": ["No Known Drug Allergies (NKDA)"],
|
|
"language": "en",
|
|
"onboarding_completed": true
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
# Architecture Sequence Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
autonumber
|
|
actor Patient
|
|
participant Frontend as React / Web App
|
|
participant API as FastAPI Server
|
|
participant Sarvam as Sarvam Indic ASR / TTS
|
|
participant Gemini as Google Gemini AI
|
|
|
|
Patient->>Frontend: Clicks "Start Intake"
|
|
Frontend->>API: POST /v1/voice-onboarding/start
|
|
API->>Sarvam: Generate Spoken Greeting (TTS)
|
|
API-->>Frontend: Returns session_id, greeting_text, audio_url, response_time_ms
|
|
Frontend->>Patient: Plays audio greeting & displays text
|
|
|
|
Patient->>Frontend: Speaks into Microphone / Types
|
|
Frontend->>API: POST /v1/voice-onboarding/{session_id}/turn (formData)
|
|
alt Voice Audio Sent
|
|
API->>Sarvam: ASR Speech-to-Text
|
|
Sarvam-->>API: Returns transcript
|
|
end
|
|
API->>Gemini: Process context & generate dynamic turn
|
|
Gemini-->>API: Returns dynamic response & auto-classified specialty
|
|
API->>Sarvam: Generate AI Spoken Response (TTS)
|
|
API-->>Frontend: Returns ai_text, audio_url, suggested_specialty, response_time_ms
|
|
|
|
Frontend->>Patient: Plays AI audio response & updates UI
|
|
|
|
Patient->>Frontend: Final Confirmation
|
|
Frontend->>API: POST /v1/voice-onboarding/{session_id}/complete ({confirmed: true})
|
|
API-->>Frontend: Returns Final Structured EHR JSON
|
|
```
|