Gokulvarthan-Ramesh/aionboarding
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
dev
aionboarding/ai-patient-onboarding/API_DOCUMENTATION.md
T

9.9 KiB
Raw Permalink Blame History

AI Patient Onboarding — Frontend Integration API Reference Guide

Base URL: http://127.0.0.1:8000
Protocol: HTTP / REST + JSON + multipart/form-data

Overview

The AI Patient Onboarding system provides two modes of integration for frontend web/mobile applications:

  1. Voice AI Onboarding API (/v1/voice-onboarding/...): Supports voice microphone recording, ASR (Speech-to-Text), synthesized audio response streaming (TTS), Google Gemini AI triage, response timing, and EHR JSON export.
  2. Text AI Onboarding API (/v1/onboarding/...): Supports pure text chat-based intake, Google Gemini AI reasoning, state extraction, and EHR JSON export.

SECTION 1: Voice AI Onboarding API (/v1/voice-onboarding)

1.1 Start Voice Onboarding Session

Initializes a voice onboarding session and returns localized greeting text + synthesized spoken audio URL.

  • Endpoint: POST /v1/voice-onboarding/start
  • Content-Type: application/json

Request Payload

{
  "patient": {
    "id": "PAT-9624",
    "name": "Gokul Varthan",
    "age": 25,
    "gender": "male"
  },
  "category": {
    "type": "general",
    "code": "general_medicine",
    "name": "General Medicine / Primary Care"
  },
  "language": "ta-IN"
}

Note: category is optional. If omitted, it defaults to general_medicine (Auto-Triage).

Supported Languages

  • ta-IN: Tamil (Pechu Tamil Spoken Dialect)
  • hi-IN: Hindi (Conversational Spoken Hindi)
  • te-IN: Telugu (Conversational Telugu)
  • kn-IN: Kannada (Conversational Kannada)
  • en-IN: Indian English (Empathetic Spoken English)

Response (200 OK)

{
  "success": true,
  "session_id": "VOICE-4C5F63F0",
  "patient_id": "PAT-9624",
  "category": {
    "type": "general",
    "code": "general_medicine",
    "name": "General Medicine / Primary Care"
  },
  "language": "ta-IN",
  "status": "in_progress",
  "stage": "greeting",
  "greeting_text": "வணக்கம் Gokul Varthan, நான் உங்க AI அசிஸ்டன்ட் நிலா. உங்களுக்கு உடம்புக்கு என்ன பண்ணுது, என்ன பிரச்சனைன்னு சொல்லுங்க?",
  "audio_url": "/v1/voice-onboarding/audio/aud_32dd00c39aa1",
  "response_time_ms": 36.33,
  "response_time_sec": 0.036
}

1.2 Submit Voice/Text Conversation Turn

Submits the patient's voice audio recording or text fallback. The backend runs ASR (Sarvam Speech-to-Text), Google Gemini AI reasoning, and returns spoken audio URL + dynamic triage category.

  • Endpoint: POST /v1/voice-onboarding/{session_id}/turn
  • Content-Type: multipart/form-data

Form Fields

Field Name Type Required Description
audio File (WebM/WAV) Optional Audio blob recorded by microphone.
text_fallback string Optional Text entered by patient. Pass null or omit when sending audio so Sarvam ASR runs.

React / JavaScript Integration Example

const sendVoiceTurn = async (sessionId, audioBlob, textInput) => {
  const formData = new FormData();
  if (audioBlob) {
    formData.append('audio', audioBlob, 'recording.webm');
  }
  if (textInput && textInput.trim()) {
    formData.append('text_fallback', textInput.trim());
  }

  const response = await fetch(`http://127.0.0.1:8000/v1/voice-onboarding/${sessionId}/turn`, {
    method: 'POST',
    body: formData,
  });

  return await response.json();
};

Response (200 OK)

{
  "success": true,
  "session_id": "VOICE-4C5F63F0",
  "patient_transcript": "எனக்கு ரொம்ப மன அழுத்தம் மற்றும் மனச்சோர்வு இருக்கு",
  "ai_text": "புரியுதுங்க கோகுல், இந்த மன அழுத்தமும் மனச்சோர்வும் எத்தனை நாளா இருக்குன்னு சொல்ல முடியுமா?",
  "stage": "reason_for_visit",
  "completed": false,
  "audio_url": "/v1/voice-onboarding/audio/aud_a1b2c3d4e5f6",
  "suggested_specialty": {
    "type": "speciality",
    "code": "psychiatry",
    "name": "Psychiatry & Mental Health Care",
    "confidence": 0.95,
    "reason": "Automatically classified based on clinical keyword: 'மன அழுத்தம்'"
  },
  "response_time_ms": 1420.50,
  "response_time_sec": 1.421
}

1.3 Stream Spoken Audio (TTS)

Streams synthesized audio response files.

  • Endpoint: GET /v1/voice-onboarding/audio/{audio_id}
  • Response Headers: Content-Type: audio/wav, Content-Disposition: inline

Audio Playback Code

const playAudioStream = async (audioUrl) => {
  if (!audioUrl) return;
  const fullUrl = audioUrl.startsWith('http') ? audioUrl : `http://127.0.0.1:8000${audioUrl}`;
  const audio = new Audio(fullUrl);
  try {
    await audio.play();
  } catch (err) {
    console.warn("Audio autoplay blocked by browser:", err);
  }
};

1.4 Complete Voice Session & Export EHR JSON

Finalizes intake and exports the complete structured EHR JSON.

  • Endpoint: POST /v1/voice-onboarding/{session_id}/complete
  • Content-Type: application/json

Request Payload

{
  "confirmed": true
}

Response (200 OK)

{
  "success": true,
  "session_id": "VOICE-4C5F63F0",
  "status": "submitted_and_cleaned",
  "message": "Voice onboarding successfully saved to patient record.",
  "result": {
    "patient_id": "PAT-9624",
    "category": {
      "type": "speciality",
      "code": "psychiatry",
      "name": "Psychiatry & Mental Health Care"
    },
    "chief_complaint": "Severe mental depression and stress",
    "symptoms": [
      {
        "name": "Mental depression",
        "duration": "2 months",
        "severity": "Moderate",
        "description": "Patient reports persistent sadness and low mood."
      }
    ],
    "medical_history": [],
    "medications": [],
    "allergies": ["No Known Drug Allergies (NKDA)"],
    "additional_information": "Referred to specialist psychiatrist.",
    "language": "ta-IN",
    "onboarding_completed": true
  },
  "response_time_ms": 2840.10,
  "response_time_sec": 2.840
}

SECTION 2: Text AI Onboarding API (/v1/onboarding)

2.1 Start Text Onboarding Session

  • Endpoint: POST /v1/onboarding/start
  • Content-Type: application/json

Request Payload

{
  "patient": {
    "id": "PAT-9624",
    "name": "Gokul Varthan",
    "age": 25,
    "gender": "male"
  },
  "category": {
    "type": "general",
    "code": "general_medicine",
    "name": "General Medicine / Primary Care"
  },
  "language": "en"
}

Response (200 OK)

{
  "success": true,
  "session_id": "ONB-8F3A2B1C",
  "patient_id": "PAT-9624",
  "category": {
    "type": "general",
    "code": "general_medicine",
    "name": "General Medicine / Primary Care"
  },
  "language": "en",
  "status": "in_progress",
  "stage": "greeting",
  "message": "Hello Gokul Varthan, welcome to CureSelect Healthcare. Could you please share what health symptoms you are experiencing today?"
}

2.2 Send Text Message

  • Endpoint: POST /v1/onboarding/{session_id}/message
  • Content-Type: application/json

Request Payload

{
  "message": "I have been experiencing a severe headache and fever since yesterday."
}

Response (200 OK)

{
  "success": true,
  "session_id": "ONB-8F3A2B1C",
  "stage": "symptoms",
  "message": "Understood Gokul. Have you taken any fever or pain reliever medication yet?",
  "status": "in_progress"
}

2.3 Complete Text Onboarding & Export EHR JSON

  • Endpoint: POST /v1/onboarding/{session_id}/complete
  • Content-Type: application/json

Request Payload

{
  "confirmed": true
}

Response (200 OK)

{
  "success": true,
  "session_id": "ONB-8F3A2B1C",
  "status": "submitted_and_cleaned",
  "message": "Onboarding successfully completed, saved to patient record, and temporary session cleaned.",
  "result": {
    "patient_id": "PAT-9624",
    "category": {
      "type": "general",
      "code": "general_medicine",
      "name": "General Medicine / Primary Care"
    },
    "chief_complaint": "Severe headache and fever",
    "symptoms": [
      {
        "name": "Headache and fever",
        "duration": "1 day",
        "severity": "Severe",
        "description": "Patient reports onset yesterday."
      }
    ],
    "medical_history": [],
    "medications": [],
    "allergies": ["No Known Drug Allergies (NKDA)"],
    "language": "en",
    "onboarding_completed": true
  }
}

Architecture Sequence Diagram

sequenceDiagram
    autonumber
    actor Patient
    participant Frontend as React / Web App
    participant API as FastAPI Server
    participant Sarvam as Sarvam Indic ASR / TTS
    participant Gemini as Google Gemini AI

    Patient->>Frontend: Clicks "Start Intake"
    Frontend->>API: POST /v1/voice-onboarding/start
    API->>Sarvam: Generate Spoken Greeting (TTS)
    API-->>Frontend: Returns session_id, greeting_text, audio_url, response_time_ms
    Frontend->>Patient: Plays audio greeting & displays text

    Patient->>Frontend: Speaks into Microphone / Types
    Frontend->>API: POST /v1/voice-onboarding/{session_id}/turn (formData)
    alt Voice Audio Sent
        API->>Sarvam: ASR Speech-to-Text
        Sarvam-->>API: Returns transcript
    end
    API->>Gemini: Process context & generate dynamic turn
    Gemini-->>API: Returns dynamic response & auto-classified specialty
    API->>Sarvam: Generate AI Spoken Response (TTS)
    API-->>Frontend: Returns ai_text, audio_url, suggested_specialty, response_time_ms

    Frontend->>Patient: Plays AI audio response & updates UI

    Patient->>Frontend: Final Confirmation
    Frontend->>API: POST /v1/voice-onboarding/{session_id}/complete ({confirmed: true})
    API-->>Frontend: Returns Final Structured EHR JSON
Reference in New Issue View Git Blame Copy Permalink