This series has been adding validators to Mellea's Instruct-Validate-Repair (IVR) loop for Qiskit, one gap at a time. The first post used flake8-qiskit-migration to catch deprecated APIs. The second put each benchmark problem's check() test in the loop, so the model could see when its code ran but gave the wrong answer.

This post covers a third gap: code that uses the right APIs, passes its tests, and still contains code smells that might silently impact the program execution.

Code smells that tests don't see

A code smell is code that runs without error but is likely to cause incorrect or unexpected behavior. Python developers have a mature linting culture for these. However, quantum programs have their own class of smells, which classical linters and unit tests are not able to catch: a gate applied after a qubit has already been measured, a register that is allocated and never used, etc.

LintQ is a static analyzer for Qiskit built on CodeQL. It models circuits, registers, and gates as CodeQL abstractions and ships queries for ten quantum-specific smells described in the LintQ paper, including:

LintQ rule	What it flags
`measure-all-abuse`	`measure_all()` used although a classical register already exists
`operation-after-measurement`	Gate applied to a qubit after it was measured
`constant-classic-bit`	A qubit is measured but was never transformed
`oversized-circuit`	Register with allocated but unused qubits
`double-measurement`	Two measurements in a row on the same qubit

LintQ is neither sound nor complete, like most linters. But it is deterministic and reproducible, and when it flags something it names the rule and the circuit involved. As the earlier posts showed, that is the property that matters for a repair loop.

Two validators, one requirement

The setup follows the earlier posts: Mellea's m.instruct() with a requirement and a MultiTurnStrategy. The difference is that the requirement's validation function now runs two checks in sequence:

from mellea.stdlib.requirements import req, simple_validate
from validation_helpers import extract_code_from_markdown, validate_correctness

# validate_lintq() is defined in the next snippet


def tests_and_lintq(problem: dict):
    def check(output: str) -> tuple[bool, str]:
        code = extract_code_from_markdown(output)
        passed, reason = validate_correctness(problem, code)
        return validate_lintq(code) if passed else (passed, reason)

    return req(
        "The code must pass the problem's test suite and raise no LintQ warnings",
        validation_fn=simple_validate(check),
    )

simple_validate hands check() the model's latest output as a string, and turns the (passed, reason) tuple it returns into a validation result whose reason becomes the repair feedback.

Correctness goes first, because there's no point linting a program that doesn't run. validate_correctness() executes the code against the problem's own test suite with a 30-second timeout. In order to keep functional correctness as a first order priority, only a program that passes the tests gets linted.

validate_lintq() writes the code to a temporary directory, builds a CodeQL database from it, runs the LintQ-all.qls query suite, and turns the SARIF results into repair feedback, one [rule-id] message line per finding:

import json
import os
import subprocess
import tempfile


def validate_lintq(code: str) -> tuple[bool, str]:
    lintq = os.environ["LINTQ_DIR"]
    with tempfile.TemporaryDirectory() as tmp:
        os.makedirs(f"{tmp}/src")
        with open(f"{tmp}/src/candidate.py", "w") as f:
            f.write(code)
        for cmd in (
            ["database", "create", f"{tmp}/db", "--language=python", f"--source-root={tmp}/src"],
            [
                "database", "analyze", f"{tmp}/db", f"{lintq}/LintQ-all.qls",
                "--format=sarifv2.1.0", f"--output={tmp}/out.sarif",
                f"--additional-packs={lintq}/qlint/codeql/src:{lintq}/qlint/codeql/lib",
            ],
        ):
            subprocess.run(["codeql", *cmd], check=True, capture_output=True)
        with open(f"{tmp}/out.sarif") as f:
            results = [r for run in json.load(f)["runs"] for r in run["results"]]

    if not results:
        return True, ""
    findings = "\n".join(f"[{r['ruleId']}] {r['message']['text']}" for r in results)
    return False, f"LintQ warnings:\n{findings}"

Wiring it into Mellea is the same m.instruct() call as before. Each problem gets its own session with a fresh ChatContext, so one task's repair conversation never leaks into the next:

from mellea import start_session
from mellea.stdlib.context import ChatContext
from mellea.stdlib.sampling import MultiTurnStrategy

with start_session(backend_name="<backend>", model_id="<model-id>", ctx=ChatContext()) as m:
    result = m.instruct(
        problem["prompt"],
        requirements=[tests_and_lintq(problem)],
        strategy=MultiTurnStrategy(loop_budget=3),
        return_sampling_results=True,
    )

Results

We ran gpt-oss-120b on the hard variant of Qiskit HumanEval, where the prompt is a natural-language instruction with no code stub, so the model also has to write the function signature. We also ran it on the full Quantum Katas set. The loop budget was three attempts: the first generation plus up to two repairs. "First pass" is generation 0 on its own; "post repair" is the outcome of the whole loop.

To reproduce these numbers, this is the setup we used:

Component	Version or setting
Model	`gpt-oss-120b`, served through an OpenAI-compatible endpoint and called with Mellea's `OpenAIBackend`
Strategy	`MultiTurnStrategy(loop_budget=3)`
Python	3.12.8
Mellea	0.7.0
Qiskit	`qiskit` 2.4.2, `qiskit-aer` 0.17.2, `qiskit-ibm-runtime` 0.45.1, `qiskit-ibm-transpiler` 0.18.0
CodeQL CLI	2.26.3
IBM Quantum account	Saved using (`QiskitRuntimeService.save_account(...)`)

Note: LintQ only runs on programs that pass their tests. If the "Flagged by LintQ" or "LintQ warnings raised" count goes up after repair, it's only because the loop fixed correctness for more programs, so more of them reached LintQ. It doesn't mean repair made the code smellier.

Qiskit HumanEval (hard), 151 tasks

Metric	First pass	Post repair
Passed both validators	24/151 (16%)	70/151 (46%)
Failed	127/151 (84%)	81/151 (54%)
Failed the test suite	113	77
Flagged by LintQ	14	4
LintQ warnings raised	17	4
Fixed by the repair loop	—	46

Qiskit Quantum Katas, 350 tasks

Metric	First pass	Post repair
Passed both validators	25/350 (7%)	163/350 (47%)
Failed	325/350 (93%)	187/350 (53%)
Failed the test suite	297	148
Flagged by LintQ	28	39
LintQ warnings raised	30	47
Fixed by the repair loop	—	138

With a budget of three attempts, the share of tasks that are both correct and smell-free went from 16% to 46% on Qiskit HumanEval (hard) and from 7% to 47% on Quantum Katas.

Takeaways

The three posts in this series use three kinds of validator, and each one catches problems the others miss:

Validator	Catches	Misses
`flake8-qiskit-migration`	Deprecated and removed APIs	Behavior
Functional tests (`check()`)	Wrong answers, crashes	Quantum-specific smells
LintQ	Quantum code smells	Behavior, and intent

None of them required changes to Mellea. Each is a function that returns (is_valid, error_message), wrapped in a requirement. The loop takes care of turning failures into repair prompts. The more validators you stack, the more aspects are checked and enforced during the generation.

A few practical notes if you want to try LintQ in your own loop:

Check correctness first. LintQ builds a CodeQL database for every candidate, which takes a few seconds. Skipping that for programs that fail their tests saves time and keeps the repair feedback focused on one problem at a time.
Keep the rule ID in the message. [ql-unmeasurable-qubits] Circuit 'qc' has more qubits (1) than classical bits (0) tells the model which rule fired and on which circuit.

Try it

LintQ is a set of CodeQL queries, so it needs the CodeQL CLI and a checkout of the LintQ repository:

Install the CodeQL CLI and put codeql on your PATH (brew install codeql works on macOS). Check it with codeql version.

Clone LintQ and install its query pack dependencies:

git clone --depth 1 https://github.com/sola-st/LintQ.git
(cd LintQ/qlint/codeql/src && codeql pack install)

Point LINTQ_DIR at the checkout:
```
export LINTQ_DIR="$(pwd)/LintQ"
```

This blog post is built as an extension of the Mellea IVR example. The full reproduction code, including the LintQ validator, will be released in the IBM open-source project CodeHelm. Adding LintQ to it means adding one more validation function with the code shown above.