Skip to content

One component. A living receptionist.

@khaveeai/react renders a VRM avatar with idle life, lip-sync and gestures, and exposes the conversation as hooks. Bring any realtime provider. This whole website is built on it.

Read the docsView on npm
$ npm install @khaveeai/react @khaveeai/core

Three steps to a talking avatar.

1

Wrap your app

KhaveeProvider holds the avatar state: VRM instance, expressions, current animation and chat status.

2

Render the avatar

Put VRMAvatar inside a react-three-fiber Canvas. Pass a .vrm file and a map of animation clips.

3

Connect a provider

Pass a RealtimeProvider to the config. useRealtime() gives you connect, sendMessage, the transcript and the live status.

The snippet below is, give or take a camera, what renders Nong Khavee on this page.

reception.tsx
import { KhaveeProvider, VRMAvatar, useRealtime } from '@khaveeai/react'
import { Canvas } from '@react-three/fiber'

export function Reception({ provider }) {
  return (
    <KhaveeProvider config={{ realtime: provider }}>
      <Canvas camera={{ position: [0, 1.3, 3], fov: 30 }}>
        <VRMAvatar
          src="/models/nongkhavee.vrm"
          animations={{ base: '/animations/idle.fbx', talk_1: '/animations/talk.fbx' }}
        />
      </Canvas>
      <Desk />
    </KhaveeProvider>
  )
}

function Desk() {
  const { connect, sendMessage, chatStatus } = useRealtime()
  return <button onClick={connect}>{chatStatus}</button>
}
provider.ts
// Swap the brain, keep the JSX.
import { OpenAIRealtimeProvider } from '@khaveeai/providers-openai-realtime'

const provider = new OpenAIRealtimeProvider({
  useProxy: true,
  proxyEndpoint: 'https://api.platform.khavee.ai/api/v1/projects/PROJECT_ID/chat/token',
  voice: 'coral',
  language: 'th',
})

Small packages, one interface.

@khaveeai/react

VRMAvatar, GLBAvatar, KhaveeProvider, useRealtime, useVRMExpressions, useAnimations, AvatarPostFX, ShadowFloor.

@khaveeai/core

RealtimeProvider and pipeline interfaces, vendor-neutral tool calling, typed project data from the platform.

@khaveeai/providers-openai-realtime

Full-duplex voice over WebRTC with automatic lip-sync and tool calls, with a token proxy so keys stay on the server.

@khaveeai/providers-mock

Canned LLM and TTS classes for tests and demos. No keys, no network.

Your UI never learns which vendor is talking.

Everything above the provider line depends only on the RealtimeProvider interface. Swap OpenAI's Realtime API for a composed STT, LLM and TTS pipeline, or for a scripted provider like the one running this page, and the JSX stays the same.

Your app

Chat UI, kiosk shell, booking screens

@khaveeai/react

VRMAvatar · KhaveeProvider · useRealtime

RealtimeProvider interface

connect · sendMessage · chatStatus · getAudioAnalyser

Providers

OpenAI Realtime · Generic STT/LLM/TTS · Scripted (this page)

The parts you'd rather not write twice.

Mixamo retargeting

Drop FBX clips from Mixamo into the animations map. Bones are remapped to the VRM humanoid automatically.

Status-driven clips

Name a clip talk, think, listen or greet and it plays for that chat status, with pose-aware crossfades.

Idle life

Breathing, weight-shift sway, blinking, expression drift and camera-relative gaze, all procedural.

Lip-sync

MFCC phoneme analysis from the provider's audio, mapped to the aa, ih, ou, ee, oh mouth shapes.

Tool calling

Register functions once. Providers map their own wire formats onto one vendor-neutral ToolCall shape.

Render quality

Shadows, anisotropy, tone mapping and an optional bloom + SMAA pass, tuned for MToon materials.

Ship a receptionist this sprint.

Grab an API key from the platform, install the packages, and bring your own VRM or use ours.

Get an API keyTalk to an engineer