Vibgyor Technologies, Oct 2024 to now

Multi-provider LLM platform

Switch between OpenAI, Gemini and AWS Bedrock with one environment variable and zero code changes.

One grading request, three providers. The responses are illustrative.

OpenAI

```json
{"criterion": "B2",
 "awarded": "2",
 "follow_through": true}
```

Wrapped in a code fence. The number is a string.

Gemini

Here is the result:
{"criterion": "B2",
 "awarded": 2,
 "followThrough": "yes"}

Prose before the JSON. camelCase key. "yes" instead of true.

Claude on Bedrock

{"criterion": "B2",
 "awarded": 2,
 "follow_through": true,
 "note": "method mark"}

An extra key the schema doesn't have.

What the product receives

{"criterion": "B2",
 "awarded": 2,
 "follow_through": true}

One shape, whichever provider answered.

1Problem

Code that calls one LLM vendor directly ties its cost and reliability to that vendor, and vendor pricing and capabilities keep changing.

Designed one interface over OpenAI, Google Gemini and AWS Bedrock (through the Converse API).
Routed every call site through that interface, so none of them names a provider.
Normalised each provider's response into the same shape before the product uses it.

Choose the provider with one environment variable.
Changing vendors is a configuration change with zero code changes.
Send non-critical tasks to cheaper models.
Cost follows how much a task matters, and call sites stay the same.
Keep vendor details out of call sites.
Pricing and capabilities change often, so provider code lives in one place.

Switching providers is a config change, application code stays the same, and cheaper models handle the non-critical work.

I can walk you through the decisions and trade-offs behind it.