Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Curve Analysis class #46

Merged
merged 85 commits into from
Jun 8, 2021

Conversation

nkanazawa1989
Copy link
Collaborator

@nkanazawa1989 nkanazawa1989 commented May 11, 2021

Summary

This PR adds CurveAnalysis class that provides basic functionalities to extract curve data and perform proper error handling for subclasses.

Details and comments

Based on this base class, we can easily implement several fitting subclasses. Data extraction and error handling is nicely performed behind the scene. This class overrides BaseAnalysis._run_analysis and provides several function-wise private methods that subclasses can override. For example _run_fitting, _create_figure, _data_processing, and _post_processing. Internal data representation CurveEntry (named tuple) is introduced to manage curve data.

To define new analysis, basically you can override base class attributes.

For example, T1 experiment fit may be written as

class T1Analysis(CurveAnalysis):
    __x_key__ = "delay"
    __fit_funcs__ = [library.exponential]
    __param_names__ = ["a", "tau", "b"]

and IRB fit may be written as

class IRBAnalysis(CurveAnalysis):
    __x_key__ = "ncliffs"
    __series__ = [
        SeriesDef(
            name="standard_rb",
            param_names=["a", "alpha_std", "b"],
            fit_func_index=0,
            filter_kwargs={"interleaved": False}
        ),
        SeriesDef(
            name="interleaved_rb",
            param_names=["a", "alpha_int", "b"],
            fit_func_index=0,
            filter_kwargs={"interleaved": True}
        )
    ]
    __fit_funcs__ = [library.exponential]
    __param_names__ = ["a", "alpha_std", "alpha_int", "b"]

As you can see subclasses don't need to write their own code. This will give us two benefits:

  • This drastically simplifies unittests and maintenance overhead (you only need to write unittest for method that you override).
  • We can easily serialize the analysis class (if method is not overridden).

Hint for review

  • Details are written in the CurveAnalysis class docstring.

  • Actual usages are shown in unittest.

  • Flow of fit operation

    1. _run_analysis is called
    2. _run_analysis calls _extract_curves.
    3. _extract_curves calls _data_processing (so that subclass can override the logic).
    4. _extract_curves generates CurveEntrys and return them.
    5. _run_analysis calls _run_fitting (so that subclass can override the logic, i.e. initial guess/weights/bounds...)
    6. _run_fitting calls _series_curve_fit
    7. _series_curve_fit performs a linearized multi-objective non-linear least squares fit and returns AnalysisResults.
    8. _run_analysis calls _post_processing followed by _create_figure.
    9. _run_analysis returns analysis results and figures

Copy link
Collaborator

@yaelbh yaelbh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it looks very good. My comments are mostly questions/hesitations and typos.

qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
Returns:
New AnalysisResult instance containing the result of post analysis.
"""
return analysis_result
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here, not sure whether to return a dictionary or an AnalysisResultV1 object

qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
Co-authored-by: Yael Ben-Haim <[email protected]>
Copy link
Contributor

@eggerdj eggerdj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some ideas on how to better deal with the data processing.

qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Show resolved Hide resolved
test/analysis/test_curve_fit.py Outdated Show resolved Hide resolved
Copy link
Contributor

@gadial gadial left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the PR is really excellent and will save some of the current complexity in RB; however, I'm not sure if it'll be possible to use RB with it without some further additions (in particular the handling of p0).

- add default data processor
- add data processor calibraiton
- integrate run_fit and series fit
- add data pre-processing (i.e. RB needs to take mean)
- add fitter setup for initial guess and other options
@nkanazawa1989
Copy link
Collaborator Author

Thanks for reviewing! @eggerdj @gadial @yaelbh
I updated the PR and I think it's ready for the second review.

@nkanazawa1989 nkanazawa1989 mentioned this pull request May 13, 2021
3 tasks
Copy link
Contributor

@eggerdj eggerdj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the training logic. I still have a few questions and comments related to it.

qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
return AnalysisResult(result)


def level2_probability(data: Dict[str, Any], outcome: Optional[str] = None) -> Tuple[float, float]:
Copy link
Contributor

@eggerdj eggerdj May 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

level2_probability can be removed, see my comment above.

qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/fit_functions.py Outdated Show resolved Hide resolved
Copy link
Contributor

@eggerdj eggerdj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good. Still a few comments and questions.

qiskit_experiments/characterization/qubit_spectroscopy.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Show resolved Hide resolved
qiskit_experiments/analysis/curve_fitting.py Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Show resolved Hide resolved
qiskit_experiments/analysis/curve_analysis.py Outdated Show resolved Hide resolved
@eggerdj eggerdj mentioned this pull request Jun 7, 2021
Merged
Copy link
Contributor

@eggerdj eggerdj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eggerdj eggerdj mentioned this pull request Jun 8, 2021
4 tasks
@eggerdj eggerdj merged commit be3e21e into qiskit-community:main Jun 8, 2021
@coruscating coruscating added this to the Release 0.1 milestone Jun 16, 2021
paco-ri pushed a commit to paco-ri/qiskit-experiments that referenced this pull request Jul 11, 2022
* wip curve fit analysis

* wip curve fit analysis

* curve_fit complete

* - unittest
- bug fix
- lint

* black

* removed redundant code

* fix unittest

* fix docstring

* fix docstring

* wording fix

Co-authored-by: Yael Ben-Haim <[email protected]>

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* Feedback from eggerdj

- add default data processor
- add data processor calibraiton
- integrate run_fit and series fit
- add data pre-processing (i.e. RB needs to take mean)
- add fitter setup for initial guess and other options

* readd 52c99ea and 0420d1d

* add fit functions

* lint

* review by eggerdj r2

- change dp calibration logic
- add gaussian function

* add data processor keys in metadata

add data processor keys in metadata

* conform to qiskit-community#41

* feedback from chris1
- more general docstring
- add default parameter to fit funcs

* feedback from chris2
- parameter dict handling

* feedback from chris3
- import path

* lint

* add fit option validation

* add unittest and integration test

* black & lint

* simplify the analysis class

* misc

* add default figure generation

* lint

* add default value

* fix docstring typo

* remove outcome from default processing options

* black and lint

* add processor training check

* fix pre processing routine

* update rb analysis as an example
- add num_qubits to curve analysis
- add label generation to curve analysis

* post process error handling when fit failed

* add axis label

* update axis formatting

* more integration with new options

* update irb

* move class attributes to options

* review comment from Chris

* fix bug causes list indices must be ...

* Analysis result formatting

* update error docstring

* lint

* bug fix
- reorder args of protected methods (series comes before xvals)
- make analysis result list
- relevant test fix
- add filter kwargs to interleaved analysis

* lint

* fix spect analysis

* fix composite analysis to use instance's analysis option

* adjust curve figure appearance

* black

* update data and option handling

* - private -> protected member
- param list generation and validation in new method
- all nan sigma handling
- zero sigma handling in curve fitter
- add data extraction utils
- allow bounds = None
-

* update spectroscopy

* black & lint

* rerun rb notebook

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* - more docstring
- add y normalization

* black

* add TODO comment

* add analysis class information to result data

* fix unittest

* update spect analysis docstring

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

* rb analysis class docstring

* format docstring

* Update qiskit_experiments/analysis/curve_analysis.py

Co-authored-by: Daniel Egger <[email protected]>

Co-authored-by: Yael Ben-Haim <[email protected]>
Co-authored-by: Daniel Egger <[email protected]>
Co-authored-by: Christopher J. Wood <[email protected]>
@nkanazawa1989 nkanazawa1989 deleted the feature/curve_analysis branch October 27, 2022 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants