Bybot is already strong. Here's what I'd do to own its next chapter.

At 80M+ users, even a 1-point shift in chatbot containment is tens of thousands of tickets that never reach a human — that's the entire business case for this role. Crypto support is also uniquely high-stakes: questions are about money in motion (stuck deposits, wrong-network transfers, withdrawal holds, P2P disputes, liquidations), they're time-pressured and emotional, and a wrong automated answer isn't just unhelpful — it can be financially harmful. That raises the bar on when Bybot should answer vs. escalate, which is the thread running through this whole audit. And it isn't abstract for Bybit: the Feb 2025 ~$1.5B cold-wallet exploit (the largest crypto theft to date) is exactly why money-movement and account-security intents must reach a human fast — getting that routing right is the heart of this role.

Six duties in the JD. Here's what each one actually means in week-to-week work, and where I cover it on this page.

Requirements call for 5–7 yrs chatbot/CS-AI ops, the CS ticket lifecycle & escalation/routing logic, EN+中文 fluency, CRM familiarity, basic NLP/AI, and strong analytical + PM skills. Fit mapped in §12.

Traced from the public help center and support docs (Jun 2026). Bybit runs a tiered self-service → assisted funnel. Each stage is a place Bybot either deflects a ticket or leaks one to a human.

Everything tagged confirmed is observable from public pages cited in §13; internals are deliberately framed as questions, not assumptions.

Bybot handles the common path well — these aren't claims that it's broken. They're the failure classes that never fully go away on any exchange CS bot, because the product keeps changing and the long tail is infinite. This is the work that's never "done," and what I'd own: measure each, watch the trend, attack the worst.

Black-box, no code access — and a rule: don't take the JD's word for it. A posting describes intended scope, not the deployed system, so every claim is a hypothesis with a confidence and a probe to confirm it. The JD narrows the search; live behaviour is the arbiter.

Confirm probes: paraphrase one intent 5 ways (rigid template reuse ⇒ intent bot; fluent variety ⇒ LLM); type 中文 on the EN page; inspect the chat widget's network origin for *.salesforce.com. Internals are framed as questions, never asserted.

Where the bar is, and what Bybot can borrow — researched from public sources (Jun 2026; cited in §13).

The JD's first duty. I treat the bot as a product with a tight build-measure-learn loop — and the analyst as its owner across Product, Eng and CS Ops.

"Analyse fallback and resolution rates" is duty #2. The trap is optimising deflection (user didn't reach a human) instead of resolution (user's problem was actually solved). Here's the metric tree I'd run.

Duty #3, and the reason fluent Mandarin is a hard requirement. The KB is built from three sources — Help Center content, internal SOPs, and patterns in CRM ticket data — and in a RAG-style bot it is the model's knowledge, so KB quality is bot quality. I'm a native-level EN + 中文 (+ BM) writer, so I can own both languages directly rather than route every Chinese edit through translation.

This is the exact loop I ran at BOB — I authored documentation and FAQs that measurably cut repeat support queries. Same mechanism here, now feeding a bot as well as a reader.

Duty #4. "QA through scenario testing" only scales if scenarios are a persistent suite, not ad-hoc clicking. Every change runs the gauntlet before it ships, in both languages.

I built a version of this as an analyst reviewing automated AI outputs at KIP Protocol. It turns "this answer feels off" into a consistent, trainable label — so QA is comparable across reviewers, languages and weeks, and the failure mix tells you what to fix next.

Each reviewed transcript gets one primary label + severity; the weekly distribution is the roadmap. Critical classes also feed the safety regression set (§08).

I built a working bilingual RAG support bot — bot.web3wagmi.com — then stress-tested it as an adversarial QA analyst and shipped five fixes in one session, each gated by an eval that must hold safety 10/10 · 0 regressions or it doesn't ship. The same loop I'd run on Bybot: read the failure → find the weak intent → fix KB/routing → prove the metric moved → don't regress it.

Each fix maps to a JD lever: precision/recall on the deflect boundary (§06), EN/中文 parity (§07), regression-gated QA (§08), escalation routing (★). The point isn’t the bot — it’s that I find where even a solid bot leaks, fix it, and prove it stayed fixed.

Frontier LLMs commoditised language understanding — so "NLP optimisation" now means the layer around the model: retrieval, routing, safety guardrails and evals. Duty #6 is "lead NLP/AI optimisation." Most CS-analyst applicants stop at dashboards; I also build the tooling. This is the shape of a regression harness that runs the golden + safety sets against the bot and blocks a release on any safety regression — illustrative of approach, not production code.

# bybot_eval.py — run the golden/safety suite, gate the release
import json, statistics as st
from bybot_client import ask        # thin wrapper over the bot API

def grade(case, resp):
    # scorecard labels from §09 → machine-checkable
    if case["must_escalate"] and not resp["escalated"]:
        return "missed_escalation"      # CRITICAL — never ship
    if case["intent"] != resp["intent"]:
        return "no_match"
    if not resp["grounded"]:            # answer cites a real KB chunk?
        return "hallucinated_policy"
    return "ok"

def run(suite, lang):
    rows = [(c, ask(c["utterance"], lang=lang)) for c in suite]
    labels = [grade(c, r) for c, r in rows]
    crit = [l for l in labels if l in ("missed_escalation", "hallucinated_policy")]
    return {
      "lang": lang,
      "containment": round(st.mean(r["contained"] for _, r in rows), 3),
      "match_rate": round(labels.count("ok") / len(labels), 3),
      "critical": len(crit),
    }

if __name__ == "__main__":
    suite = json.load(open("golden_safety.json"))
    report = [run(suite, l) for l in ("en", "zh")]   # both languages, every run
    for r in report: print(r)
    # release gate: any critical failure in any language → fail the build
    assert all(r["critical"] == 0 for r in report), "safety regression — blocking release"

The point isn't this exact script — it's that I think about chatbot quality as something you can measure, gate and regress like software, in EN and 中文 together. I also build onchain AI agents and read SDKs/contracts, so I'm comfortable wherever the bot meets engineering.

I don't know the internal stack, so this is vendor-agnostic: the frontier tools I'd evaluate per JD responsibility. ✓ shipped = I've already run a working version (✦ Proof). The posture: instrument first, measure the real gaps, then pick buy / build / tune against the numbers — the data picks the dish, not the menu.

Claims are split into confirmed (verified from public pages) and illustrative/inferred (modelled or from the JD) throughout. Code is illustrative of approach, not production config. This is an unsolicited audit prepared as interview homework — happy to walk through any section live.

Prepared by Edward Tay · for the Bybit Client Service Analyst — (Senior) AI Chatbot role · Jun 2026 · edwardtay.com · Edwardtay7@gmail.com