Перейти к содержанию

KYC Service — Technical Reference

Overview

Node.js microservice that performs OCR and face-matching for KYC (Know Your Customer) verification. It receives job requests from Serverpod Auth Center via HTTP, processes them asynchronously using BullMQ, and writes results back to the kyc_verification table in PostgreSQL.

Architecture

Worker Flow

Technology Stack

Component Technology
Runtime Node.js 20+
HTTP server Express 4
Job queue BullMQ 5
Queue backend Redis (ioredis)
OCR / AI Gemini API (@google/generative-ai) or Ollama (local)
Face matching CompreFace (ArcFace embeddings via HTTP)
Storage (images) Garage / S3-compatible (@aws-sdk/client-s3)
Database PostgreSQL (pg)

Process Model

A single Node.js process runs both components:

  1. Express HTTP server — accepts POST /api/v1/process, enqueues jobs, returns 202 immediately
  2. BullMQ Worker — started in-process at server startup, processes jobs from the kyc-jobs queue
  3. Watchdog — setInterval every 5 minutes finds kyc_verification records stuck in in_review status and marks them ocr_failed

Processing Pipeline

For each job the worker executes these steps in order:

  1. Generate presigned S3 URLs — for passportKey, selfieKey, and optionally stampKey (5-minute expiry)
  2. Gemini OCR — sends passport image to Gemini model, extracts: name, date of birth, ID number, expiry. Returns structured JSON.
  3. OCR validation — if all returned fields are null, throws UnrecoverableError('INVALID_PHOTO:passport:...') — no retry
  4. Face matching (CompreFace) — downloads both images, sends to CompreFace POST /api/v1/verification/verify. Returns { score, reasoning } where score is ArcFace cosine similarity (0.0–1.0)
  5. Face match threshold check — if score < FACE_MATCH_THRESHOLD (default 0.75), throws UnrecoverableError('FACE_MISMATCH:...') — no retry
  6. Write to PostgreSQL — calls updateKycResult(kycId, payload) which encrypts OCR data with PII_ENCRYPTION_KEY and writes to kyc_verification.ocrResult (bytea), sets status = 'ocr_complete'

On any failure after all retry attempts: calls updateKycFailed(kycId, errorMessage) which sets status = 'ocr_failed'.

AI Provider Selection

The OCR step uses a pluggable provider selected by KYC_PROVIDER env var:

KYC_PROVIDER Provider Model
gemini (default) Google Gemini API gemini-3-flash-preview
ollama Local Ollama gemma4 (or OLLAMA_MODEL)

Both providers implement the same interface: extractOcr() and matchFaces().

Note: CompreFace is used exclusively for face matching — it is not affected by KYC_PROVIDER. The Gemini/Ollama matchFaces() implementation exists but CompreFace is the production path (src/services/facerecog.js).

Retry Policy

  • Max attempts: 5
  • Backoff: exponential, starting delay 15 seconds → 30s → 60s → 120s → 240s
  • BullMQ job deduplication: jobId = "kyc-{kycId}" — duplicate submissions are silently ignored
  • UnrecoverableError: used for INVALID_PHOTO and FACE_MISMATCH — BullMQ skips all remaining retries immediately
  • Gemini 429 rate limit: caught and rethrown as Error('RATE_LIMIT:{waitSec}:...') — triggers normal backoff retry
  • Concurrency: 3 (three jobs processed in parallel)

Status Values

Status Meaning
in_review Job enqueued or in-progress
ocr_complete Processing succeeded — OCR data + face match written
ocr_failed Processing failed after all retries, or unrecoverable error

Watchdog

Runs every 5 minutes (and once at startup). Finds all kyc_verification records where: - status = 'in_review' - submittedAt < NOW() - KYC_STUCK_TIMEOUT_MIN minutes (default 15) - reviewedAt IS NULL

Sets them to ocr_failed with failureCode = 'processing_timeout'.

PII Encryption

OCR results (which include real name, ID number, DOB) are encrypted before database storage using PII_ENCRYPTION_KEY. The same key must be set in the Serverpod backend (passwords.yaml: piiEncryptionKey). See src/services/pii_crypto.js.

Key Files

File Purpose
src/index.js Express server, BullMQ queue, watchdog setup
src/worker.js BullMQ Worker — full processing pipeline
src/services/ai.js AI provider selector (Gemini or Ollama)
src/services/gemini.js Gemini OCR + face match implementation
src/services/ollama.js Ollama OCR + face match implementation
src/services/facerecog.js CompreFace face matching (production path)
src/services/s3.js Presigned URL generation (Garage/S3)
src/services/db.js PostgreSQL writes (updateKycResult, updateKycFailed, markTimedOutKyc)
src/services/prompts.js Gemini prompt templates (OCR_PROMPT, FACE_PROMPT)
src/services/pii_crypto.js AES encryption for OCR result blobs