An academic career-guidance system for Karachi secondary students. It reasons about which degrees a student can genuinely enter, how competitive they would be, and where the honest answer is that the data does not exist.
| Degree | Eligible | Closing merit | Standing |
|---|---|---|---|
| Programme A | Yes | 76.4% | Strong fit |
| Programme B | Yes | 81.2% | Reach |
| Programme C | No | 68.0% | Subject not held |
| Programme D | Yes | not published | confirm with the university |
Row four is the reason this project exists. Many universities never publish a closing merit. The system leaves that cell empty and says so, rather than filling it with a plausible number a student would act on.
A student in Karachi choosing a degree is making one of the least reversible decisions of their life, usually with no access to professional counselling. The hard part is not recommendation. It is reasoning truthfully about eligibility across multiple examination boards, five academic streams, equivalence rules for international qualifications, professional gating from three separate regulatory councils, and merit mechanics that differ per university and are often not published at all.
The admission data is richest for exactly the students who already have the most options. Building only where the data is good means building only for them.
Showing a student more possibilities is helpful right up to the moment one of them is a degree a regulator will never let them enter.
Every significant decision in this system resolves one of those two tensions.
Eligibility, competitiveness, and ranking are computed in plain deterministic code with no model in the loop. The language model receives what was already decided and explains it to the student. A wrong recommendation is therefore a bug with a line number, not a prompt that behaved differently on a Tuesday.
It also bounds the blast radius of the least trustworthy inputs. The most-validated signals drive the ranking. The softer, less-validated ones shape only how the result is worded, and cannot override it. The worst case for the weakest input is awkward phrasing, not a wrong degree.
The conversational layer explains what the engine decided, answers follow-up questions about cost and prospects, and adapts to how each student writes, including Roman Urdu and mixed script. It never decides eligibility, competitiveness, or rank.
None of these were designed on a whiteboard. Each replaced something already running and already wrong.
Eligibility follows the foundational subject a student studied, read against the governing regulations, rather than being inferred from how capable they appear.
Degrees with no published closing merit carry an explicit state saying so, and point the student at the admissions office. Cutoffs are derived from official merit lists or left empty.
Assessment is driven by the intersection of what a student studied and what their reachable degrees actually test.
Pass thresholds change over time. A student who passed under an older rule is never re-judged against a newer one.
The ranked set always includes a guaranteed minimum of aspirational targets that sit above the student's current standing but are genuinely eligible.
The reasoning engine is a directed graph of independent steps rather than a sequence of prompts. Steps that do not depend on each other execute concurrently, so the work that can overlap does.
Every step runs on every turn and there are no conditional edges. An activation pass makes irrelevant steps no-op instead of removing them from the graph, so every join is always satisfied. A stalled turn caused by an unmet dependency is not a bug that was fixed. It is a state the graph cannot enter.
Every step reads and writes a single typed structure rather than passing loose dictionaries. No step can invent a field another step expects, and a shape change surfaces at the boundary instead of three steps downstream as a silent default.
Responses stream incrementally so the student sees reasoning as it arrives. Under live conditions on the deployed service, a turn held 106.3 seconds past a 30-second proxy timeout without dropping the connection, verified against the production deployment rather than a local run.
Eligibility logic is written against a schema, not against institutions. A new university inherits every gate, filter, and merit rule the moment its data conforms. The tradeoff is recorded honestly: the logic is schema-dependent, so a missing field silently widens eligibility rather than failing loudly, which is a known hazard rather than a solved one.
Any change that is not meant to alter what students see is verified to produce identical output for existing profiles. Improvements are additive by default, and a restructure has to be argued for rather than assumed.
The pipeline was originally sequential and took 20 to 25 seconds before a student saw anything. Independent work was separated to run concurrently and several steps were split into finer sub-steps, so less of the work sat on the critical path. First response now arrives in 3.7 seconds, with the rest streaming in behind it.
All seven instruments and the full 265-item session exist in both English and Roman Urdu, the register students actually write in. The conversation follows whichever the student uses, including mixed script mid-sentence. This was built in from the start rather than added as a translation layer, because a psychometric item that reads awkwardly measures something other than what it was validated to measure.
The full report is generated by a program: seventeen chapter modules, a diagram generator, a shared formatting layer, and an audit pass, compiled into a single document. Figures and the text referencing them cannot drift apart, and the whole thing is reproducible from source rather than being a file three people take turns editing.
Collection, daily refresh, classification, aggregation, and orchestration, each a separate stage with atomic writes and resumable progress. The source would not expose one attribute directly, so the collector recovers it through multiple filtered passes cross-referenced by record id. 14,595 titles were mapped to canonical fields and aggregated across 51 of them, with the records it skipped and the fields that received nothing both recorded rather than quietly dropped.
Prompts are written so that anything working on the weakest supported model works unchanged on a stronger one. Changing which model runs a given step is a configuration edit, not a rewrite, which is what made it possible to develop against one provider and deploy on another.
A guard that never fires is indistinguishable from a guard that does not exist, and worse, because you now trust something that is not working. This principle shaped more of the system than any feature did.
One gate strips claims the system has no measurement to support. To prove the gate was doing the work rather than a neighbouring check, it was fed a plausible, carefully hedged sentence that passed every other quality criterion. Stripped when unsupported, preserved when supported. A controlled experiment, not an assumption.
The internal architecture reference is built by reading the running system. Where the source records nothing it prints a refusal rather than a plausible sentence, and it displays its own age, because an earlier version once looked confidently current while sitting 151 commits behind.
A planned feature depended on output varying with how firmly an instruction was phrased. A controlled probe held everything else constant and tested a light and a firm version of the same instruction. The firm one produced less of what was wanted, inverting the intended gradient, so the feature was dropped before it was built. A gradient the model cannot produce is theatre.
A single call cannot be told to use some context for one purpose but not another. So the call that decides never receives the material at all, and a second call applies it to the already-decided output. That turns a property you have to test for into one the structure guarantees, because the deciding call's input is identical to a known-good baseline.
Not a comment. Purpose, fully typed inputs and outputs, formal step-numbered pseudocode rather than a paste of the code, complexity, all three classes of invariant, enumerated edge cases, and the alternatives rejected with the reason. Written only after the component is complete, because a document written mid-implementation records the design you started with and then misleads with authority.
An audit of the boundary between app and service found five real defects that 209 passing tests had missed, because every fixture seeded the canonical values the code expected and never the payload the live app actually sends.
Corrections are additive. Stale text is never deleted or quietly reworded, so the record contains cases where a correction was itself wrong and had to be corrected back. Evidence behind the assessment design is graded on a seven-point scale by how close each source sits to Karachi students of this age, and every step away from that population is stated rather than absorbed.
Severity in this project is measured in wrong guidance to a student, not in code quality. This is a selected record, newest first. Dates are real; nothing here is reconstructed.
There is no holistic review of counselling quality and no outcome data. Building it needs labelled outcomes that do not exist yet.
Finished items are tagged either live-exercised or reasoned-from-code. Of 55 completed items, 18 have been exercised against a live run. The remaining 37 are believed. The tagging exists specifically so that suspicion stays pointed at the right set.
When a student volunteers a rich personal history, the reply can retell it as though it were the system's own experience. Four instances, all inside the one case that volunteered detail, none in the other four. The existing guard misses it because it checks claims about the student, and this is a false claim about the speaker.
Result cards are sent to the screen before the record is written. If the write fails the student sees their results, the error is swallowed, and the dashboard is empty on next launch.
The job-market pipeline runs and produces a per-field demand history. Nothing in the ranking reads it yet.
A change made three days earlier produced wrong guidance for one group of students.
The way one question bank was assembled meant roughly 32% of commerce students would have been asked a duplicate.
Re-verifying one university against its published documents found a legal minimum stored as 50% where the source says 60%, an incomplete list of mandatory subjects, and fees that had been entered as estimates.
A change made the day before, intended to keep resumed answers in sync, was traced through its consumers and shown to cause silent scoring corruption on submit.
A scripted answer leaned on a large 2024 study that could not be found. It traced back to a single non-peer-reviewed analysis from 2023.
An earlier pass concluded the reasoning over-asserted when data was thin. A later pass found the opposite: the thinnest case is the most honest, because the component declines outright, and over-assertion is triggered by conflicting data instead.
A live probe found a class of gates that executed on every turn regardless of their condition, then wrote and persisted false ran-or-skipped entries. Reading the code would not have found it. Only running it and comparing the record against reality did.
An audit of the seam between app and service found a naming mismatch that both deflated merit and triggered a wrong regulatory exclusion, and a scale mismatch that had left counselling branches dead for every live student.
It assumed 25 results per page. The source returns nine or ten, so it skipped every position past the first ten of each page. No error, no warning, and a perfectly plausible dataset.
A prior data session had written merit history values that did not match any published source.
A pre-demo gate check found a knowledge file empty, which collapsed the match component of every degree's score to the same constant.
The evaluation is five rather than more for a reason worth stating: at a measured 10.6 model calls per turn against a daily quota, roughly 42 turns per day were possible. Depth was chosen over breadth deliberately.
Two independent structural studies, roughly a thousand students between them, found the circular ordering of the underlying interest model does not fit this population. The instrument's psychometrics were sound and interest-to-aspiration correspondence held for several careers, but the geometry did not. The weighting was deliberately not changed, because the limitation is an absence of local structural validation rather than a weak signal, and down-weighting the best-validated input in favour of less-validated ones would make the system less defensible. It is disclosed instead, and the robustness comes from architecture: no recommendation rests on that geometry alone.
Original items built on established frameworks. First-party reliability on a Karachi sample is not established and will be computed as data accrues. Support is not uniform across them, and the weakest is named as such rather than averaged into the rest.
The system models how families in Karachi actually weigh degree status, because a recommendation that ignores it will be rejected regardless of fit. Those values were assigned from general market knowledge and have not been validated by a survey.
The source, the curated admission data, and the derived eligibility rules are private. The curation is the work: months of reconciling regulations, prospectuses, and published merit lists into something a system can reason over correctly.
Everything on this page is reasoning rather than mechanism. I am glad to walk through the architecture in a conversation.
Or write to ma.sharif.waqas@gmail.com.