OpenAI FDE technical interview practice: implementation and evaluation

Use the vacancy’s work to choose the evidence

These are 15 original practice questions, inferred from OpenAI’s verified job descriptions and public guidance checked on 2026-10-05. They are not official questions, leaked tasks, a confirmed Tokyo FDE loop or an internal rubric. The facts article retains each fact’s role, location and source scope. Related jobs provide comparison; they do not share a proven hiring process.

Several questions use common engineering practices. Their connection to OpenAI is the named vacancy’s responsibility and regional scope. Tokyo FDE and Applied AI Engineer both need personal implementation evidence. The Codex role adds developer-workflow evaluation; the Architect role adds account-level design decisions. Product references supplement preparation and are explicitly labelled; they do not prove a hiring requirement. The general engineering guide covers quality dimensions, but its format examples do not establish your assigned interview. Confirm tools, language and AI permissions for each round.

Use a real, shareable project where one exists. Describe a proposal as a proposal and an unexecuted test as unexecuted. Never adopt hypothetical numbers as your past results or OpenAI’s thresholds. An honest account of a failed trial can supply better evidence than an unexplained improvement claim.

The original preparation map below distinguishes three artifacts. Arrows represent your rehearsal sequence, not OpenAI’s process. Implementation traces show what happened; an evaluation record shows how it was judged; a decision record explains the consequence.

1. 1Owned implementation and request path
2. 2observable failure or result
3. 3evaluation case with a defined metric

1. 1Evaluation comparison and constraints
2. 2decision record
3. 3deployment, revision or hold

1. 1Unexpected production case
2. 2classified failure
3. 3updated evaluation coverage

Prepare one artifact from each row, with customer identifiers removed. That gives a follow-up a concrete place to go: code boundary, measurement or decision. The questions below ask for different evidence; they should not all be answered with the same “we improved accuracy” story.

OAI-TQ01: Show your own production implementation

Grounding facts: OAI-T01, OAI-T08, OAI-A02, OAI-A05, OAI-I08.

Role / region: Tokyo FDE; Tokyo Applied AI Engineer. Engineering guide is general, not an FDE-specific rubric.

Official support: Tokyo FDE job · Tokyo Engineer job · General hiring guide.

Practice question — inferred, not an official question: Choose a real customer-facing system. Which frontend and backend changes did you personally implement, and how did those changes make the workflow usable in production?

What the question probes — inference: The job descriptions make hands-on contribution consequential. The inferred aim is to distinguish implementation judgment from coordination or a team-level success story, and connect a code change to customer behavior. This is not a claim about OpenAI’s scoring rubric.

Evidence, numbers and decisions to include:

• Draw the request path: user action, frontend state, backend validation, model/tool call, persistence and displayed result. Mark the components you owned and the ones built by others.
• Name an actual code-level decision, a rejected alternative and the constraint that decided it. Explain a test or observed failure that would have invalidated your design.
• Use shareable measurements: successful completions/attempts, a latency percentile over a stated period, or a classified defect count before and after. State units, denominator, comparison and your contribution; do not invent a percentage improvement.

Follow-up 1: Which failure crossed the frontend/backend boundary?

Answer points: Explain whether the server completed an operation while the client timed out, what durable state or request ID allowed diagnosis, and how retry behavior avoided a duplicate effect. If that was not your incident, describe it as a proposed test and state the expected result.

Follow-up 2: What did your review change beyond making the demo run?

Answer points: Show the review concern, resulting change and evidence: input validation, maintainable boundaries, performance or meaningful tests. Separate a reviewer’s decision from yours and explain what remained unverified at release.

Avoid a shallow answer: “We built an AI chatbot and customer satisfaction improved.” It hides the implementation boundary, measurement and your own decision. A hypothetical architecture is useful only when clearly labelled proposed.

OAI-TQ02: Integrate the model without losing the business transaction

Grounding facts: OAI-T01, OAI-T04, OAI-A02, OAI-A03.

Role / region: Tokyo FDE; Tokyo Applied AI Engineer. Integration/recovery design is an original exercise, not a specified test.

Official support: Tokyo FDE job · Tokyo Engineer job.

Practice question — inferred, not an official question: A customer’s existing workflow already has an authoritative record. Where would you insert a model call, and how would you keep partial completion from becoming an incorrect business result?

What the question probes — inference: The inferred aim is to test whether integration is understood as an end-to-end state transition, including systems the candidate cannot rewrite. The postings support integration and stable operation, not any particular transaction architecture.

Evidence, numbers and decisions to include:

• Name the system of record, input contract, authorized writer and states such as pending, completed or failed. Distinguish a model suggestion from an accepted business update.
• Show timeout, malformed response and downstream-write failure paths. Explain where validation happens, what a retry repeats and what must happen once; identify the existing team’s constraints.
• For a real system, report observed failed/duplicate completions over attempts and recovery time with the period. For a proposal, list controlled failure fixtures and expected durable states without presenting them as measured outcomes.

Follow-up 1: What if the model succeeds but the customer database write fails?

Answer points: Identify which result may be retained, what remains pending and which authorization must be checked again. Compare replaying the model call with reusing a validated result; account for freshness, duplicated effects and cost.

Follow-up 2: What if the customer cannot change its legacy interface?

Answer points: Keep the constraint explicit. Consider an additive integration boundary with a bounded schema and failure state, explain the operational cost, and identify a point at which the project should wait rather than conceal an unsafe interface.

Avoid a shallow answer: “Call the API and add retries.” It leaves the authoritative record, partial completion and duplicate side effects undefined.

OAI-TQ03: Translate model behavior into product behavior

Grounding facts: OAI-T09, OAI-A03, OAI-T08.

Role / region: Tokyo FDE; Tokyo Applied AI Engineer.

Official support: Tokyo FDE job · Tokyo Engineer job.

Practice question — inferred, not an official question: A model’s output is plausible but wrong on some inputs. What should the customer’s product display, permit and record when confidence in the result is insufficient?

What the question probes — inference: Model experience in the FDE posting includes its effect on product experience. The inferred aim is to connect uncertain output to a user action and failure consequence, instead of treating prompt quality as the entire product.

Evidence, numbers and decisions to include:

• Choose a concrete task and distinguish a suggestion, a draft and an executed action. Define what evidence the user sees, the applicable permission and the state when evidence is missing.
• Classify an observed error by consequence: wrong answer, wrong recipient, unsafe action or abandoned workflow. Explain the design response and the extra friction it imposes.
• Use task-completion and correction/abandonment observations with denominators alongside model quality. A numerical confidence display is not calibrated merely because a model produced it.

Follow-up 1: When would you require human confirmation?

Answer points: Compare reversibility, impact, evidence quality and current authorization. State exactly what is confirmed and by whom, how approval expires when the input changes, and what the user can correct before execution.

Follow-up 2: What if confirmation makes adoption worse?

Answer points: Measure where users leave and which high-consequence errors confirmation prevents. Consider reducing the action’s scope or automating deterministic checks; do not remove a control using adoption alone as the justification.

Avoid a shallow answer: “Use a better model and a disclaimer.” It does not define behavior at the point where the customer’s work could be harmed.

OAI-TQ04: Build evaluation coverage from customer work

Grounding facts: OAI-A04, OAI-T02, OAI-E01, OAI-E03.

Role / region: Tokyo Applied AI Engineer; Tokyo FDE evaluation-feedback context. The evaluation guide is a technical reference.

Official support: Tokyo Engineer job · Tokyo FDE job · Evaluation reference.

Practice question — inferred, not an official question: How would you turn a customer workflow into a representative evaluation set, and decide what evidence is sufficient to compare two implementations?

What the question probes — inference: The inferred aim is to connect the Engineer’s systematic evaluation requirement to customer tasks and FDE field insights. The question asks for coverage and a decision, not a universal target accuracy or a known interview exercise.

Evidence, numbers and decisions to include:

• Define task success, users and input categories before collecting cases. Include common traffic, consequential rare failures and data-quality differences; record sampling and permitted data use.
• Separate development cases from a stable comparison set. Record model/prompt/retrieval/tool versions, metric definition, case count and actual repeated observations where variability matters.
• Compare the baseline and candidate on the same cases. Report denominators and failures by slice, plus latency/cost when relevant; identify what the sample cannot establish.

Follow-up 1: What if aggregate quality improves but a critical slice regresses?

Answer points: Name the affected workflow and harm, check the sample and scorer, and compare a restricted rollout, repair or hold. Explain who agrees to the acceptance condition rather than inventing a company-wide threshold.

Follow-up 2: How do new production cases enter the set?

Answer points: Describe authorized/redacted capture, failure categorization, expected behavior and review ownership. Preserve a fixed comparison set while tracking a changing task distribution; record when a case represents a new requirement.

Avoid a shallow answer: “Use a public benchmark and 100 examples.” Neither the benchmark nor an arbitrary count establishes this customer’s coverage or acceptance conditions.

OAI-TQ05: Calibrate automated graders with human judgment

Grounding facts: OAI-A04, OAI-E02.

Role / region: Tokyo Applied AI Engineer. Official evaluation guidance supplements the role description.

Official support: Tokyo Engineer job · Evaluation reference.

Practice question — inferred, not an official question: When a grader says two answers are equally good but customer reviewers disagree, how would you investigate and change the evaluation?

What the question probes — inference: The inferred aim is to test measurement validity. The posting names graders and human judgment; a trustworthy evaluation needs an explanation of what a score actually measures and where it fails.

Evidence, numbers and decisions to include:

• Write a task-specific rubric in your own words: correctness, supported evidence, completeness and prohibited behavior where applicable. Distinguish preference from failure severity.
• Retain a shareable disagreement sample and independent reviewer labels. Report agreement/disagreement counts by category, review instructions and ambiguous cases; do not label reviewer preference as ground truth without examining it.
• Test whether the grader rewards superficial style, verbosity or unsupported confidence. Explain the corrected scoring rule, its comparison results and the unresolved limitations.

Follow-up 1: What if humans disagree with each other?

Answer points: Identify whether the task definition is ambiguous or the reviewers lack needed context. Resolve the requirement with the customer owner, retain disputed cases and uncertainty, and repeat a blinded review under the clarified instructions.

Follow-up 2: Can a model-based grader verify business completion?

Answer points: Separate textual judgment from deterministic facts such as persisted state or the correct transaction recipient. Combine a grader with observable system assertions; explain which failure cannot be detected from the answer alone.

Avoid a shallow answer: “A stronger model can grade it.” That changes the tool without demonstrating that the score matches the customer’s decision.

OAI-TQ06: Evaluate a tool-using agent at its action boundary

Grounding facts: OAI-A03, OAI-E04, OAI-F02.

Role / region: Tokyo Applied AI Engineer. Frontier access descriptions are product context, not an asserted customer stack.

Official support: Tokyo Engineer job · Evaluation reference · Frontier product context.

Practice question — inferred, not an official question: An agent produces a good final answer but called the wrong tool or passed an unsafe argument. How would you design evaluation and control for that case?

What the question probes — inference: The inferred aim is to separate answer quality from action correctness. The technical guide explicitly includes tool choice and arguments, while the role covers tools and governance; the proposed controls are the candidate’s design.

Evidence, numbers and decisions to include:

• Specify intended tool, schema, target resource, caller identity and allowed scope. Check authorization using current system state rather than trusting the model’s text.
• Create cases for the wrong tool, malformed arguments, another customer’s resource, repeated execution and revoked access. Define a safe terminal state and a trace event for each.
• Measure valid authorized actions/attempted actions and prohibited effects separately from final-answer quality. Describe actual tests if executed; otherwise retain them as expected results.

Follow-up 1: Where should a dangerous write be stopped?

Answer points: Locate the last deterministic boundary before the side effect. Describe schema and resource authorization there, and any customer-approved confirmation; a prompt warning alone is not a runtime guarantee.

Follow-up 2: How would you test a revoked permission during a long task?

Answer points: Insert a controlled revocation before a later tool call. Explain the permission refresh, denied state, user-visible result and audit record; avoid claiming a specific vendor behavior unless that endpoint has been checked.

Avoid a shallow answer: “The answer looked right, so the agent passed.” It ignores what the agent actually changed and for whom.

OAI-TQ07: Decide whether multiple agents are justified

Grounding facts: OAI-A03, OAI-E05, OAI-E03.

Role / region: Tokyo Applied AI Engineer; evaluation guide as technical support. No multi-agent interview requirement is claimed.

Official support: Tokyo Engineer job · Evaluation reference.

Practice question — inferred, not an official question: A customer proposes specialized agents for every subtask. What evidence would make you keep one agent, add a deterministic stage or introduce a handoff?

What the question probes — inference: The inferred aim is to justify complexity with observed failure mechanisms. The guide supports evaluation-grounded architecture decisions; adding agents is not itself evidence of engineering maturity.

Evidence, numbers and decisions to include:

• Describe a simple baseline and the failure it cannot address. Explain the proposed specialist’s responsibility, its input/output contract and the context it actually needs.
• Compare architectures on identical tasks: correct routing, successful completion, transfer errors, elapsed time and total cost. Record retries and handoffs so an apparent improvement does not hide extra work.
• Choose using the customer’s constraints and actual observations. Include operational owners, debug difficulty and a condition for removing a specialist if it adds little value.

Follow-up 1: What counts as a handoff failure?

Answer points: Include the wrong recipient, lost task state, dropped constraints and circular transfers. State the expected receiving state and recovery path, then show the trace that distinguishes transfer failure from task failure.

Follow-up 2: When is a deterministic stage preferable?

Answer points: Consider a stable schema, explicit rules and actions whose correctness can be checked directly. Explain the limits of that stage and which genuinely ambiguous judgment remains, using the same evaluation cases for comparison.

Avoid a shallow answer: “More agents are more accurate.” It supplies no failed baseline, transfer contract or measured benefit.

OAI-TQ08: Cover multilingual and adversarial failures

Grounding facts: OAI-E06, OAI-A04, OAI-T11.

Role / region: Tokyo Applied AI Engineer; Tokyo FDE bilingual context. Evaluation guidance is not a published language-test rubric.

Official support: Evaluation reference · Tokyo Engineer job · Tokyo FDE job.

Practice question — inferred, not an official question: A workflow works on short English examples. How would you test Japanese input, long context, multiple intentions and instructions embedded in retrieved data?

What the question probes — inference: The inferred aim is coverage beyond the easiest demonstration. The sources support edge-case evaluation and bilingual customer work; they do not specify a Japanese adversarial hiring task.

Evidence, numbers and decisions to include:

• Start with the real workflow’s language and document distribution. Include equivalent intentions across languages, domain terms, mixed input and relevant context lengths; do not use translation alone as proof of coverage.
• For each adversarial or multi-intent case, specify evidence that should be treated as data, instructions that are authorized, allowed tools and the expected safe result. Use synthetic inputs when customer data cannot be used.
• Report failures by language, input class and consequence with case counts. Check whether scoring itself is weaker in Japanese; separate model, retrieval, authorization and grader failures.

Follow-up 1: What if Japanese scores are lower because the reference answer is poor?

Answer points: Review the expected answer with a qualified domain/language reviewer, retain the reasoning and correct the case without moving the goalposts for a favored model. Re-run the same candidates against the revised record.

Follow-up 2: How do you know a prompt injection was safely handled?

Answer points: Inspect tool intentions and actual effects, not only a refusal string. Verify no unauthorized access/write, explain the terminal state, and retain a redacted trace sufficient to reproduce the case.

Avoid a shallow answer: “Translate the English tests and add a safety prompt.” It ignores task distribution, scoring validity and actual unauthorized effects.

OAI-TQ09: Trade latency and cost against successful work

Grounding facts: OAI-A03, OAI-A02.

Role / region: Tokyo Applied AI Engineer.

Official support: Tokyo Engineer job.

Practice question — inferred, not an official question: The customer finds the system too slow and costly. How would you identify the expensive path and choose an optimization without losing the task’s required quality?

What the question probes — inference: The role names latency and cost alongside reliability and safety. The inferred aim is to expose an engineering choice using a workload, rather than optimizing token count without considering completed work.

Evidence, numbers and decisions to include:

• Break elapsed time into retrieval, model, tools, network and retries using observed traces. State workload, concurrency, period and latency percentiles; average time alone may hide a poor tail.
• Define cost per attempted task and per successful task, currency/billing units, measured usage and exclusions. Use current pricing only if verified; do not present a made-up vendor price as evidence.
• Compare a constrained change: smaller context, fewer calls, a model choice or deterministic processing. State freshness/privacy limits for caching and test quality and failure behavior on the same cases.

Follow-up 1: What if a cheaper model requires more retries?

Answer points: Compare total usage and success over the full task, including timeout and repair effort. Identify the retry ceiling and which errors should not retry; a lower per-call price is insufficient.

Follow-up 2: What if users need a quick acknowledgement but work takes longer?

Answer points: Separate acknowledged, pending and completed states. Explain a proposed asynchronous design’s persistence, cancellation, visibility and recovery; measure user-perceived completion separately from server work time.

Avoid a shallow answer: “Use the cheapest model and cache everything.” It leaves quality, retries, data freshness and access boundaries untested.

OAI-TQ10: Observe enough to diagnose, without collecting everything

Grounding facts: OAI-A03, OAI-A02, OAI-F03.

Role / region: Tokyo Applied AI Engineer. Frontier logs are product context; proposed tracing is not an OpenAI retention policy.

Official support: Tokyo Engineer job · Frontier product context.

Practice question — inferred, not an official question: A customer reports that the AI workflow sometimes fails, but the demo cannot reproduce it. What observations would let you identify the responsible layer?

What the question probes — inference: The inferred aim is to connect observability to repairable failures. The role names observability and debugging; the candidate must decide which trace is necessary and what should not be collected.

Evidence, numbers and decisions to include:

• Define a correlation path across client request, backend, model/tool and durable outcome. Record version/configuration, timing, status and a bounded error class; distinguish an attempted call from an effect.
• List what is omitted or redacted, retention purpose, access owner and investigation procedure. Explain how you can reproduce an authorized minimal case without retaining every prompt or customer document.
• Show one actual diagnosis: symptom, trace-supported hypothesis, disconfirming test and corrected observation. Report incident counts or recovery duration only with defined scope and period.

Follow-up 1: How do you distinguish provider failure from application failure?

Answer points: Compare request/response status, retry history, application validation and downstream state. Retain the minimal evidence needed for escalation, distinguish confirmed cause from suspicion, and verify the user-visible result.

Follow-up 2: What if the trace contains customer secrets?

Answer points: Explain containment and access with the customer/security owner, redaction and an appropriate authorized preservation decision. Redesign the capture contract and test it; do not copy sensitive traces into an interview example.

Avoid a shallow answer: “Log all prompts and watch the dashboard.” It confuses data collection with an investigation path and ignores privacy cost.

OAI-TQ11: Make the permission boundary reviewable

Grounding facts: OAI-T03, OAI-R04, OAI-F02.

Role / region: Tokyo FDE cross-functional work; Tokyo Applied AI Architect design scope. Frontier is an illustrative product source.

Official support: Tokyo FDE job · Tokyo Architect job · Frontier product context.

Practice question — inferred, not an official question: Before a customer agent can read documents and update a business system, what design evidence would you bring to GRC, Security and the customer’s owners?

What the question probes — inference: The FDE posting names these partners and the Architect posting names governance. The inferred aim is a reviewable authority/data path, not a claim that an FDE independently grants approval or that a particular compliance framework is mandatory.

Evidence, numbers and decisions to include:

• Map principal, resource, read/write action and enforcement point. Separate user permission, app/service credentials and agent intent; define least necessary access using the customer’s actual workflow.
• Provide a data inventory, trust boundary, failure/abuse cases and unresolved assumptions. Identify who can accept a residual risk and what evidence or configuration remains a launch dependency.
• Offer executable negative checks for another tenant, expired/revoked access and attempted privilege escalation. Record denied effects and audit evidence rather than claiming certification from a diagram.

Follow-up 1: What if the model receives permission instructions in a document?

Answer points: Treat retrieved content as evidence, not an authority source. Explain where resource access is checked using trusted identity and policy, and show a test that cannot change scope through generated arguments alone.

Follow-up 2: What if Security’s review blocks the proposed launch?

Answer points: Separate the missing evidence from an unacceptable boundary. Propose a narrower read-only or synthetic-data phase if authorized, and keep the unresolved approval as a dependency instead of treating the demo as acceptance.

Avoid a shallow answer: “Enterprise AI is secure by default.” It supplies no principal, resource scope, enforcement point or decision owner.

OAI-TQ12: Separate training use, storage and application access

Grounding facts: OAI-R04, OAI-P01, OAI-P02, OAI-P03, OAI-P04.

Role / region: Tokyo Applied AI Architect privacy design; supporting privacy source only within its named products/endpoints. Not an interview requirement.

Official support: Tokyo Architect job · Privacy reference.

Practice question — inferred, not an official question: A customer assumes that “not used for training” means no storage and unrestricted use of connected apps. How would you clarify the requirements before choosing a design?

What the question probes — inference: The inferred aim is to avoid collapsing different data controls into one reassuring phrase. The official privacy page is technical/business evidence, not a hiring policy or a blanket promise covering every endpoint.

Evidence, numbers and decisions to include:

• Separate training use, vendor retention, application logs, connected-app permissions and deletion requirements. Draw where each copy can exist and who controls it.
• Use the privacy facts with scope: default no training with opt-in exceptions; encryption does not settle retention; API retention/eligibility has endpoint and feature conditions. Confirm the actual product, endpoint and contract rather than promise zero storage.
• Record customer data classes, permitted processing, required deletion/access behavior and an owner for unresolved terms. Validate connected-app authorization and any local log policy independently.

Follow-up 1: Would encryption alone satisfy a no-retention requirement?

Answer points: Explain confidentiality during storage/transport separately from whether data exists and for how long. Identify the relevant endpoint’s actual retention mode and eligibility; consult the responsible security/legal owner for acceptance, without giving legal advice.

Follow-up 2: What can remain even with an eligible zero-retention configuration?

Answer points: Inspect the full customer application: local logs, caches, database records, third-party tools and user downloads. Keep vendor endpoint claims scoped and design each other copy’s access, deletion and verification separately.

Avoid a shallow answer: “Data is not trained on, therefore there is no privacy risk.” It confuses training, storage, access and the customer’s own application behavior.

OAI-TQ13: Keep evaluation useful as dependencies change

Grounding facts: OAI-A02, OAI-E03, OAI-X01.

Role / region: Tokyo Applied AI Engineer evaluation harnesses; official deprecation notice is dated platform context, not the hiring process.

Official support: Tokyo Engineer job · Evaluation reference · Platform retirement notice.

Practice question — inferred, not an official question: How would you keep evaluation evidence comparable while changing a model, prompt, retrieval pipeline or evaluation platform?

What the question probes — inference: The inferred aim is to separate a durable evaluation contract from a particular hosted product. As of the check date, the Evals platform retirement was planned, not completed; its notice does not withdraw evaluation as a practice.

Evidence, numbers and decisions to include:

• Preserve task definitions, authorized datasets, metric/grader versions and baseline results. Track implementation changes separately so a changed scorer is not mistaken for a better model.
• Design a migration comparison on shared cases, with missing fields, grader differences and result exports checked explicitly. State provenance and which historical results are no longer directly comparable.
• Use the dated notice accurately: 2026-06-03 is the relevant section’s announcement date; 2026-10-31 read-only and 2026-11-30 shutdown were future plans on 2026-10-05. Verify current instructions before planning a real migration.

Follow-up 1: What if the new grader changes the score scale?

Answer points: Retain both scales, compare pairwise task outcomes and human-reviewed disagreements, and establish a new baseline where needed. Do not silently stitch old and new scores into one trend.

Follow-up 2: What is your go/no-go for the migration?

Answer points: Specify data export integrity, reproducible case execution, scoring equivalence or explained differences, access controls and an owner. Show a tested or proposed rollback/hold path rather than relying on the announced shutdown date alone.

Avoid a shallow answer: “Evals is ending, so evaluation is obsolete.” It confuses a hosted platform with the engineering practice the roles require.

OAI-TQ14: Debug inside a customer environment with clear ownership

Grounding facts: OAI-T04, OAI-A02, OAI-W02.

Role / region: Tokyo FDE; Tokyo Applied AI Engineer. Customer-side coding in the SF FDSWE posting is a regional comparison, not a Tokyo obligation.

Official support: Tokyo FDE job · Tokyo Engineer job · SF software-engineering job.

Practice question — inferred, not an official question: A prototype worked in your environment but fails after integration with the customer’s stack. How do you investigate collaboratively and choose the smallest safe fix?

What the question probes — inference: The inferred aim is practical debugging with constrained access and multiple owners. The job evidence supports embedded work and hands-on debugging; it does not grant unrestricted access to a customer’s production environment.

Evidence, numbers and decisions to include:

• Write the environmental difference: identity/network, input shape, dependency/configuration versions, data permissions and runtime state. Compare a minimal authorized reproduction before changing several variables.
• Identify ownership and change permission for each component. Explain what you can inspect, what customer engineers must run and what evidence can leave the environment.
• Show hypothesis, disconfirming test, selected fix and regression check. Use actual reproduction rates, error counts or recovery time only with workload and period; explain the customer’s handoff documentation.

Follow-up 1: What if you cannot access the failing production data?

Answer points: Ask the authorized owner to produce a redacted symptom/trace or synthetic reproduction under agreed rules. State what this loses, test hypotheses on permitted inputs and hold conclusions that require unavailable evidence.

Follow-up 2: What if the quickest patch bypasses a customer control?

Answer points: Describe the risk and control owner, compare a bounded alternative or temporary pause, and make any exception an explicit authorized decision. Include expiry/removal and verification; do not hide the bypass as a technical detail.

Avoid a shallow answer: “It worked locally; the customer environment is the problem.” It avoids diagnosis, authority boundaries and responsibility for integration.

OAI-TQ15: Measure Codex workflow improvement beyond generated code

Grounding facts: OAI-C01, OAI-C02, OAI-C03, OAI-C04.

Role / region: Applied AI Engineer, Codex — Tokyo. Do not transfer its specialist requirements to every FDE posting.

Official support: Tokyo Codex Engineer job.

Practice question — inferred, not an official question: For a development team using an AI coding tool, how would you prove that the workflow improves without increasing review burden or unsafe changes?

What the question probes — inference: The Codex role covers developer tasks, automated graders, production signals and feedback. The inferred aim is to connect tool use to engineering outcomes rather than equate generated lines or subjective enthusiasm with productivity.

Evidence, numbers and decisions to include:

• Select representative tasks such as implementation, testing or review from the real team. State task difficulty, allowed context/tool use and expected repository/test behavior; keep confidential code within its permitted boundary.
• Compare time to an accepted change, first-pass test/review outcomes, rework and escaped defects over a stated task set. Separate author time, reviewer time and total lead time so work shifted to reviewers is visible.
• Combine automated checks with developer feedback and observed production results. State a limitation, a tool-assisted failure and your own view of where the tool helps or should be constrained; do not claim universal gains.

Follow-up 1: What if tests pass but the change is hard to maintain?

Answer points: Review the requirement, module boundaries and unexplained complexity, then compare a simpler change. Explain the human review finding and a task-level maintainability criterion; passing tests is evidence of covered behavior, not every quality dimension.

Follow-up 2: What if the pilot attracts only enthusiastic developers?

Answer points: Report selection bias and task mix, include a broader permitted sample and comparable baselines, and record onboarding/help time. Narrow the conclusion to the tested group until wider evidence exists.

Avoid a shallow answer: “The tool wrote more code, so productivity doubled.” It lacks accepted-output quality, reviewer work and a measured comparison.

Rehearse the decision, then test what you could not explain

This is an unexecuted preparation workflow. Pick three questions that match the exact vacancy. First write a one-page boundary diagram and measurement record; then answer aloud and have a colleague ask both follow-ups. Record every claim you could not support. Repair the missing evidence, narrow the claim or label it proposed before repeating the rehearsal.

Keep a small comparison sheet: configuration/version, representative inputs, metric definition, actual observed result, failure category and resulting decision. When numbers cannot be disclosed, use permitted ranges or normalized comparisons with their limitations. Never use confidential source code or customer records as interview material. An exercise that fails safely is still useful evidence when its limits are explained.

The delivery practice article connects these technical artifacts to customer scope, adoption and field feedback. Confirm the current vacancy and round instructions before applying; this practice set does not resolve the unknown interview format.