MerLin 0.4: Grimoire

[CassNot]

MerLin v0.4: Grimoire

Overview

MerLin 0.4 is a major step toward practical photonic machine learning workflows.
It makes models easier to assemble, simulations more faithful to hardware, and
execution paths clearer from local experiments to QPU-oriented runs.

The release adds ready-to-use models in merlin.models, including the
QORC-inspired ReservoirClassifier, the
photonic GAN PhotonicGenerator, and the
photonic QCNN QCNNClassifier. These models move
common research patterns into reusable PyTorch modules, so users can spend less
time rebuilding plumbing and more time testing photonic learning ideas.

QuantumLayer also becomes more hardware-aware. The new noisy SLOS simulation
path lets users train under realistic source, circuit, and photon-emission noise
when inference is intended to run on a QPU. This matters because training only
against ideal simulations can produce parameters that look good numerically but
fail once hardware noise is present. By exposing noise during training, MerLin
helps users optimize models for the conditions they actually expect at
inference time.

State preparation is now more explicit through EncodingSpace. Users can choose
how an amplitude tensor represents a logical state, including full Fock,
unbunched, dual-rail, partitioned, and QLOQ-style encodings. StateVector then
validates and embeds that logical state into MerLin's canonical Fock basis,
removing a common source of silent shape and ordering mistakes.

Execution is more flexible as well. MerlinProcessor can now use a Perceval
processor directly through processor=, including local Perceval processors such
as SLOS and CliffordClifford backends. The same entry point also supports remote
processors, while session-based providers remain available. This keeps local
simulation, backend comparison, and remote execution inside the same Merlin
processor workflow.

Finally, 0.4 removes several compatibility paths that were deprecated in 0.3.
The supported APIs are now more explicit: computation space belongs in
MeasurementStrategy, amplitude inputs belong in forward(), and legacy
boolean flags such as no_bunching are no longer accepted.

New Features

ReservoirClassifier

ReservoirClassifier is the new ready-to-use QORC model inspired by the
photonic reservoir computing work. It builds a
frozen photonic reservoir from a Haar-random interferometer and trains only a
classical linear readout.

This makes the QORC workflow available as a reusable model instead of requiring
users to manually wire preprocessing, reservoir feature extraction, caching,
and readout training. The model supports optional scikit-learn dimensionality
reduction, reservoir feature normalization, cached embeddings, and direct
creation of PyTorch datasets for the readout.

Typical workflow:

• create a ReservoirClassifier;
• call fit_reservoir(X) on the training inputs;
• use make_dataset(X, y) or transform_reservoir(X) to train or inspect the
  readout data;
• call predict(X) for logits on new inputs.

PhotonicGenerator

PhotonicGenerator is a PyTorch module for latent-to-sample generative
workflows based on the photonic generative model. It
wraps one or more QuantumLayer heads and delegates the final classical
interpretation to an output adapter.

This provides a cleaner base for photonic GAN-style generators. Instead of
manually feeding latent variables into a quantum layer and reshaping raw
measurement distributions for each experiment, users can define a generator
with a latent dimension, one or more quantum heads, and an adapter such as
ImageAdapter or VectorAdapter.

The generator exposes sample_latent, measure, generate, and normal
PyTorch forward behavior.

QCNNClassifier

QCNNClassifier adds a staged photonic QCNN model for image-like inputs,
following the adaptive state-injection QCNN proposal.
Architectures are built from validated QConv, QPool, and QDense stages.
Users can provide their own stage sequence or rely on the default validated
architecture.

This gives users a higher-level image-classification model without requiring
them to assemble every quantum convolution, pooling step, and dense readout by
hand.

Memristive Phase Shifters

CircuitBuilder now supports memristive phase shifters through
add_memristive_ps(...), following the idea explored in
recent photonic hardware work.

For machine-learning workflows, this gives a photonic layer a stateful
component that can carry information across repeated calls or timesteps. That
is useful for temporal dependence, recurrent-style models, feedback circuits,
and sequence experiments where the current circuit response should depend on
previous inputs rather than only on the current tensor.

The builder declares the memristive component, and QuantumLayer owns the
runtime state, history, serialization hooks, reset behavior, and
detach_memristive_state(...) helper. This moves a previously manual feedback
layer pattern into the normal builder-layer contract.

Noisy SLOS Simulations

QuantumLayer now supports a noisy SLOS path driven by pcvl.NoiseModel.
MerLin can account for the main hardware-relevant noise sources directly in
simulation:

• brightness;
• transmittance;
• phase imprecision;
• phase error;
• indistinguishability;
• multi-photon emission through g2;
• distinguishability of extra g2 photons.

Noisy simulations return probabilities. Source noise and stochastic phase
errors are represented as probability mixtures, so MerLin combines probability
distributions rather than returning ideal amplitudes.

For g2 and source-noise cases, outputs may span multiple photon-number
sectors. These outputs are represented through SectoredDistribution and
SectorResult, then converted back into tensors when required by the selected
measurement strategy.

EncodingSpace

EncodingSpace describes how the last dimension of an amplitude tensor maps
into MerLin's canonical Fock basis. It makes logical amplitude inputs explicit
and removes the need to manually enumerate Fock states for common encodings.

Supported encodings include:

• EncodingSpace.FOCK for amplitudes already in full Fock order;
• EncodingSpace.UNBUNCHED for collision-free photonic states;
• EncodingSpace.DUAL_RAIL for qubit-like binary logical states;
• partitioned encodings through EncodingSpace(modes_per_photon=[...]);
• QLOQ-style grouped encodings through EncodingSpace.qloq(...).

StateVector.from_tensor(..., encoding=...) validates the logical tensor,
embeds it into Fock order, and returns a StateVector that can be passed to
QuantumLayer.forward().

MerlinProcessor Local Processor Support

MerlinProcessor now accepts Perceval AProcessor objects through the
keyword-only processor= argument.

This means local Perceval processors can be used from the same Merlin execution
interface as remote processors and sessions. Local processors use a local
backend route. RemoteProcessor instances passed through processor= are
normalized to the remote backend route. The older remote_processor= argument
still works as a deprecated compatibility path.

This is useful when users want to evaluate Merlin models on local Perceval
backends, including local noisy or sampling backends, without leaving the
Merlin model execution workflow.

Improvements

Measurement and Output Handling

The measurement stack has been tightened around explicit output contracts.
MeasurementStrategy now owns the computation-space choice, and probability
outputs can be post-processed through readouts such as occupancy grouping.

Photon-loss, detector, and noisy source outputs are handled more consistently,
including outputs that span multiple photon-number sectors. Partial measurement
manual construction also validates measured and unmeasured mode partitions more
strictly.

Kernel and Feature-Map Cleanup

The kernel stack has been updated around the newer QuantumLayer execution
path and clearer deprecation boundaries. FeatureMap and FidelityKernel
have expanded coverage for builder metadata, angle encoding, state inputs, and
newer computation-space behavior.

Older kernel builder compatibility paths remain documented as deprecated where
they still exist.

StateVector and Sparse State Workflows

StateVector and probability-distribution objects have improved support for
sparse tensors and logical-to-Fock mappings. This helps amplitude-input
workflows stay memory-aware, especially when the logical state is much smaller
than the full Fock basis.

Documentation

The documentation has been expanded for the new 0.4 workflows:

• encoding spaces and StateVector.from_tensor(..., encoding=...);
• noisy SLOS simulations and the supported NoiseModel fields;
• photonic QGAN workflows using PhotonicGenerator;
• QCNN examples;
• QORC and ReservoirClassifier workflows;
• local and remote execution through MerlinProcessor;
• updated migration notes for removed 0.3 compatibility APIs.

Several reproduced-paper and example notebooks were also updated or added,
including photonic QGAN, QCNN MNIST, QORC tutorials, neural network embedding,
HQPINN, NQE, and CACIB bankruptcy prediction.

Breaking Changes

no_bunching removed

The legacy no_bunching flag is no longer accepted on public layer and kernel
entry points.

Use explicit computation-space configuration instead:

• MeasurementStrategy.probs(computation_space=ComputationSpace.UNBUNCHED)
• MeasurementStrategy.probs(computation_space=ComputationSpace.FOCK)

Constructor computation_space removed

Passing computation_space directly to QuantumLayer is no longer supported.
Computation space now belongs in the measurement strategy.

Legacy MeasurementStrategy access removed

Legacy enum-style measurement access such as
MeasurementStrategy.PROBABILITIES, MeasurementStrategy.MODE_EXPECTATIONS,
and MeasurementStrategy.AMPLITUDES now fails with a migration error.

Passing measurement strategies as strings, such as "PROBABILITIES", is also
no longer supported. Use the factory methods:

• MeasurementStrategy.probs(...)
• MeasurementStrategy.mode_expectations(...)
• MeasurementStrategy.amplitudes(...)
• MeasurementStrategy.partial(...)

Constructor amplitude encoding removed

QuantumLayer(..., amplitude_encoding=True) is no longer accepted.

Amplitude inputs should now be passed to QuantumLayer.forward() as either a
StateVector or a complex tensor.

Tensor constructor input_state removed

Passing a raw torch.Tensor as constructor input_state is no longer
accepted. If a state object is needed at construction time, build it explicitly
with StateVector.from_tensor(...).

Deprecations

The remote_processor= argument on MerlinProcessor is deprecated. New code
should pass the same Perceval processor through processor=.

Kernel compatibility helpers such as KernelCircuitBuilder,
FidelityKernel.simple(...), and related simple-factory compatibility
arguments are deprecated and kept only for transition.

Compatibility and Maintenance

MerLin 0.4 updates the dependency floor to perceval-quandela>=1.2.1 and
allows newer PyTorch versions up to torch<=2.12.0.

The test suite has been expanded substantially around noisy SLOS, g2 behavior,
encoding spaces, state-vector inputs, deprecation removals, MerlinProcessor
local execution, photonic generators, QCNNs, and QORC reservoir workflows.

[CassNot]

@claude review

[claude]

Claude finished @CassNot's task in 7m 0s —— View job

---

Claude review — MerLin 0.4: Grimoire

Reviewed the core source changes (≈100 files, 46k additions) across state/encoding, the noisy SLOS path, the measurement stack, QuantumLayer/builder, MerlinProcessor, the new models, and kernels/deprecations. The release is large and generally well-structured, with strong test coverage and clear migration errors for the removed 0.3 APIs. Below are the findings worth acting on before merge, ordered by severity. I verified the two HIGH items directly against the code.

---

🔴 HIGH

1. MerlinModule.uid is a class attribute → shared by every instance (remote-execution cache collision)
merlin/algorithms/module.py:57 defines uid = uuid.uuid4() at class scope, and QuantumLayer never sets an instance self.uid. MerlinProcessor uses it as the layer-config cache key:

# merlin/core/merlin_processor.py:987,994
cache = self._layer_cache.get(layer.uid)
...
self._layer_cache[layer.uid] = {"config": config}

All QuantumLayer instances therefore share one UID. When two distinct layers run through the same processor, the second silently reuses the first layer's cached export_config() (circuit, input state, param order) → wrong remote-execution results, no error.
Fix: assign per-instance in MerlinModule.__init__ (self.uid = uuid.uuid4()), or back it with an instance field. Fix this →

2. Memristive state not moved by .cuda()/.cpu()/.float() (device/dtype split)
QuantumLayer overrides to() to move memristive_state/memristive_history, but nn.Module.cuda()/.cpu()/.float()/.double()/.half() route through _apply, not to(). Since the memristive tensors are plain Python lists (not registered buffers), those move paths leave them on the old device/dtype, and the next forward() mixes devices → runtime error or silent wrong-device compute.
Fix: override _apply (or register the state/history as buffers) so all torch move paths convert the memristive tensors. Fix this →

---

🟠 MEDIUM

3. Noisy g2 path allocates on self.device/self.dtype, not unitary.device — merlin/pcvl_pytorch/noisy_slos.py:238-258,316. p_emit/weight_k and each SectorResult.tensor are built on self.device, while the sub-graph probs live on unitary.device. A CUDA unitary on a graph constructed with default device=None (without a prior .to()) raises a cross-device error. The non-g2 path correctly derives device from unitary.device (:768) — mirror that here.

4. Unguarded renormalization on possibly-zero mass — merlin/pcvl_pytorch/noisy_slos.py:790. output_probs / output_probs.sum(dim=1, keepdim) has no zero/near-zero denominator guard; OBB weights are intentionally un-normalized so this is always exercised. A degenerate/underflowing batch element yields NaN/Inf that silently propagates into the sector accumulation. Clamp the denominator or assert finiteness.

5. Occupancy readout silently renormalizes; probs() does not — merlin/measurement/readouts.py:124-126. _OccupancyReadout.forward rescales grouped probs to sum=1, while the non-occupancy probs() path (strategies.py:147-158) does not. Under photon loss / detector mass-loss the two paths diverge in semantics with no signal to the user. Make occupancy grouping a mass-preserving column sum, or normalize both paths consistently and document the intent.

6. Memristive layers unusable with batched amplitude input — merlin/algorithms/layer.py:1121-1154. For amplitude/StateVector forward there are no classical params, so parameter_batch_dim == 0 and batch_dim falls back to 1; with amplitude batch B>1 the per-memristor state stays size 1 while output.shape[0]==B, tripping the shape check at :1373-1379. Derive batch_dim from the actual input batch (including amplitude inputs).

7. to() dtype contract — amplitude_encode hardcodes complex64 — merlin/models/qcnn.py:927-930. A QCNNClassifier.to(torch.float64) feeds complex64 state vectors into float64 quantum layers → dtype mismatch / silent down-cast. Derive the complex dtype from the module's parameter dtype.

8. No seed / torch.Generator for latent sampling — merlin/models/photonic_generator.py:544-576 (sample_latent/generate/NormalLatent.sample all hit global torch.randn). Generated batches can't be reproduced without mutating global RNG. Thread an optional generator through.

9. Post-fit reconfiguration silently discards training — merlin/models/reservoir_classifier.py:443-453. _rebuild_quantum_layer → _invalidate_fit_state() resets the trained readout + cached features with no warning when noise/strategy/n_modes is changed after fit_reservoir. Warn when invalidating an already-fitted model.

10. Session path builds and discards a RemoteProcessor at construction (resource leak + new side effect) — merlin/core/merlin_processor.py:569-583. The ISession path now eagerly calls session.build_remote_processor() just to read name/available_commands, then drops _init_rp without closing it (the RPC handler holds an HTTP session/connection pool). This also introduces construction-time network/auth work that didn't exist on main. Store and close the init RP, or extract capabilities without building one.

11. compute_new_memristive_ps_angles swallows the real error — merlin/algorithms/layer_utils.py:986-996. Broad except Exception around the user update_rule re-raises a generic "does not follow the correct build", hiding genuine shape/device bugs in an otherwise-correct rule (contradicts AGENTS.md "do not hide failures"). Narrow the catch or chain the original cause.

---

🟡 LOW / cleanup

• Memristive state in state_dict as lists of tensors — layer.py:2211-2222. Non-standard Dict[str,Tensor] contract; round-trips in-house but breaks generic consumers (load_state_dict(strict=True), DDP/FSDP). Prefer stacked tensors / registered buffers.
• from_tensor partitioned dims not validated against the encoding contract — state_vector.py:214-219. Explicit n_modes/n_photons are returned unchecked; a mismatch surfaces later from inside embed as a confusing error rather than at the documented point.
• Dead code: pcvl_sampler_g2 (measurement/process.py:137-244, never wired in — also has an UnboundLocalError path if sectors is empty), resolve_photon_loss_kernel (measurement/photon_loss.py:407), reinsert_modes (models/qcnn.py:696,702,711), and the unreachable max_shots_per_call is None branch (merlin_processor.py:1150-1154).
• MeasurementStrategy.__eq__ still accepts "PROBABILITIES"/"probabilities" strings — strategies.py:458-459. String strategies are otherwise rejected; this equality leg can mask incomplete migrations.
• Stored per-call probs retain autograd graph — noisy_slos.py:761 (self._probs_per_obb_state = ...) pins the last forward's tensors/graph on a long-lived cached object; appears unused — drop it or detach.
• resolve_photon_loss not robust to its own type signature — photonic_loss.py:378-393 raises KeyError/TypeError on an empty or None-valued dict (not currently reachable, but the signature allows it).
• Broad except Exception in FeatureMap._encode_x — kernels.py:467 silently falls back when a user encoder raises (deprecated path, but still a hidden failure).
• reservoir_classifier.py:311 rejects float/'mle' n_components despite the docstring claiming any sklearn decomposition estimator works.
• Validated-correct (no action): merlin/utils/sparse.py deletion is safe (zero references repo-wide); all removed 0.3 APIs raise clear migration errors and never silently accept removed inputs; logical→Fock index mapping (FOCK/UNBUNCHED/DUAL_RAIL/partitioned/QLOQ) matches Combinadics descending-lex order; g2 emission-probability inversion and binomial sector weights are physically correct and tested against Perceval; fidelity-kernel symmetry/diagonal/PSD projection are correct; local-vs-remote routing in MerlinProcessor is sound (isinstance(RemoteProcessor) checked before is_remote); frozen reservoir is genuinely frozen.

---

Summary: Two HIGH issues (#1 shared uid and #2 memristive device moves) are correctness bugs I'd fix before release. The MEDIUM set is mostly about device/dtype robustness, silent renormalization/normalization divergences, and the session-path resource leak. Nothing structural blocks the release — the architecture and migration story are solid.
· branch release/0.4

[github-actions]

✅ Coverage Report

Overall Coverage: 83.93%

Coverage Status

• Target: 80% (warning threshold)
• Current: 83.93%
• Status: ✅ Meeting target

---

Coverage reports are informational and do not block PRs.
Generated using pytest-cov

[ben9871]

Some issues but not blocking. Can be fixed in 0.4.1