Phase 1 is infrastructure and baseline—not a new foundation model.
BrokenGPT is an India-built product and API layer for configured third-party or open-weight language models. The current phase establishes the account, chat, streaming, API-key, metering, model-disclosure, and evaluation path. It gives later model changes a measurable baseline.
This distinction matters for honest LLM research. Serving an existing model is product engineering; changing a checkpoint is model development; training a foundation model is a different scale of work. BrokenGPT will not use one label as evidence for another.
The central question has two axes, not one.
Can a model refuse fewer lawful requests without losing correctness, instruction following, or calibrated uncertainty? A refusal-rate improvement alone is not a capability result. The research program is designed to keep willingness and quality visible as separate measurements.
Refusal and over-refusal
Which lawful topics trigger a refusal, deflection, lecture, or incomplete answer?
Accuracy after answering
When the model engages, is the result correct, relevant, secure, and appropriately uncertain?
Indic and Hinglish behavior
Do scripts, transliteration, code-switching, and local context change refusal or answer quality?
Serving tradeoffs
How do precision, quantization, context length, throughput, latency, and cost change the observed result?
A result is only useful when the test conditions travel with it.
Evaluation releases are intended to include enough context to audit or repeat the claim:
- prompt text, task category, expected behavior, and inclusion or exclusion rules;
- public model identifier plus exact checkpoint or provider revision where disclosure is permitted;
- system prompt, sampling settings, maximum output, context construction, and serving precision;
- automatic metrics alongside human-review instructions, rater counts, and disagreement handling;
- latency, token use, serving cost, failed requests, and infrastructure constraints;
- limitations, contamination risk, selected examples, counterexamples, and negative results.
A polished demo is not a benchmark. A benchmark is not a deployment guarantee. The build journalwill separate engineering notes, exploratory findings, and completed evaluation releases.
Three phases, with different evidence required at each one.
| Phase | Target state | Evidence before the claim |
|---|---|---|
| Phase 1 — live | Serve configured open-weight or third-party models through one metered product path. | Working chat/API path, usage accounting, model disclosure, baseline prompt set. |
| Phase 2 — target Oct 2026 | Evaluate lower-refusal variants at the highest practical fidelity, including non-quantized serving where infrastructure permits. | Checkpoint provenance, paired refusal/accuracy results, settings, costs, regressions, limitations. |
| Phase 3 — target early 2027 | Develop a BrokenGPT-native model. | Training decisions, data governance, capability and safety evaluations, limitations, deployment card. |
Dates are targets, not guarantees. Research schedules move when evidence is weak, infrastructure changes, or a method fails. Those changes belong in the public record instead of being hidden behind a launch date.
Indian LLM research needs multilingual, locally legible evidence.
“Works in India” cannot be inferred from an English benchmark or a company location. Useful evaluation must make language coverage, scripts, code-switching, cultural context, access cost, and deployment assumptions visible. BrokenGPT’s planned India-facing work starts with a sourced map of the ecosystem and then moves toward original Indic and Hinglish prompt sets.
The first landscape note distinguishes Indian foundation-model programs from Indic-language research and from India-built application layers like BrokenGPT. Future evaluations will be published only when the underlying prompts, settings, and scoring record are ready.