Skip to content

First version of iterativerobotpy.py #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

MikeStitt
Copy link

A draft to get progress going.

Copy link
Member

@virtuald virtuald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks pretty sane.

kTest = 4


class IterativeRobotPy(wpilib.RobotBase):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is a draft, but adding docstrings can be annoying. Recommend either using the sphinxify webserver (https://github.com/robotpy/devtools/blob/main/sphinxify_server.py) to convert javadocs, or just grab the docstrings from the .pyi file that ships in our existing wheels on pypi.

Obviously, before we can merge this, all public methods need docstrings.

def testExit(self):
pass

# todo @Deprecated(forRemoval=true, since="2025")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we deprecate things, we typically only do it in the docstring.

wpilib.DriverStation.refreshData()
self._m_watchdog.reset()

self._m_word.refresh() # todo from Java implementation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C++ doesn't have this, I filed a bug to fix that (wpilibsuite/allwpilib#7889).

For now, just do self._word = DSControlWord() and it should have the same effect. Leave a TODO note to switch to refresh when it gets implemented.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, robotBase has a GetControlState method which returns (enabled, autonomous, test)... which I think would be better to use here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@auscompgeek AI tells me that "The eyes emoji (👀) generally indicates a sense of observation, attention, or interest, often with a subtle hint of intrigue or admiration. It can also be used to suggest that someone is being watched or is deliberately drawing attention to something."

What are you thinking here?

# private void printLoopOverrunMessage() {
# DriverStation.reportWarning("Loop time of " + m_period + "s overrun\n", false);
# }
wpilib._impl.report_error.reportWarning(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is correct.

I would do at the top of this file from wpilib import reportWarning. Every time there's a . in python it's another dictionary lookup, and the rio is really slow.

Copy link
Member

@auscompgeek auscompgeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some additional early feedback.

self.driverStationConnected()

# If mode changed, call mode exit and entry functions
if self._m_lastMode != mode:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Python enums are singletons, so prefer using is like the recommendation in the Python docs for enum

@MikeStitt
Copy link
Author

Top level questions:

Is the filename right?

Is the file in the right spot in the tree?

Is the class name right?

What are the appropriate tests?

What happens if someone deletes the watchdog? Is the there a higher level watchdog that will stop the motors?

Can somebody figure out how to call the shuffleboard?

@virtuald
Copy link
Member

virtuald commented Apr 6, 2025

I'm not sure about the filename/classname yet. I think you're approaching it as an alternative, but I sorta want a replacement? But, I guess we could go either way.

TBH, before deciding on alternative/replacement, probably need someone to do some testing on a real robot to see if it makes loop overrun issues more/less/same.

What are the appropriate tests?

... yeah, the only real way to test this is likely going to be by testing examples, which is something I'd like to do but haven't taken the time to do (see #154). Java has tests for TimedRobot, but they basically do the same thing that the pyfrc test framework does (maybe that's an argument for moving some of that into here...).

What happens if someone deletes the watchdog? Is the there a higher level watchdog that will stop the motors?

If someone deletes the watchdog, that's their problem -- but also, the motor watchdog is actually a separate thing? This one just tracks loop overruns and tells the user about it.

Can somebody figure out how to call the shuffleboard?

https://robotpy.readthedocs.io/projects/robotpy/en/stable/wpilib.shuffleboard/Shuffleboard.html#wpilib.shuffleboard.Shuffleboard.update

@MikeStitt
Copy link
Author

This version simulates a TimedRobot, advances time, logs to AdvantageScope in a reasonable way, but does not move the robot in simulation.

It feels like some sort of enable interlock on the simulator is missing.

Generally all status results from hal.xxx calls seem to result in a strange value, but the calls seem to work.

./rdev.sh ci run
and
./rdev.sh develop

do not seem to make the pure python files editable in place.

Could use some detailed advice on how to get the python files to be editable in place.

@MikeStitt
Copy link
Author

This is the code I'm using to simulate: https://github.com/spiresfrc9106/spiresRobot2025/blob/exp-timedrobotpy/robot.py#L46

@virtuald
Copy link
Member

virtuald commented Apr 8, 2025

For develop to work, you have to make sure that all of the packages aren't installed first. pip/setuptools don't do a great job of having an editable/non-editable package installed.

@MikeStitt
Copy link
Author

MikeStitt commented Apr 8, 2025

It's hard to tell which libraries need to be uninstalled for mostrobotpy and which need to stay for robotpy:

t2.txt:

robotpy-apriltag
robotpy-cscore
robotpy-hal
robotpy-halsim-ds-socket
robotpy-halsim-gui
robotpy-halsim-ws
robotpy-romi
robotpy-wpilib-utilities
robotpy-wpimath
robotpy-wpinet
robotpy-wpiutil
robotpy-xrp
pip uninstall robotpy
pip install robotpy
pip uninstall -r ~/t2.txt -y
pip uninstall pyntcore
pushd ~/mostrobotpy
./rdev.sh develop
 popd
mikestitt@peas mostRobotPySpires2025 % pip list
Package                   Version                   Editable project location
------------------------- ------------------------- -------------------------------------------------------------------------------
altgraph                  0.17.4
anyio                     4.4.0
appdirs                   1.4.4
astroid                   3.0.3
bcrypt                    4.2.0
black                     25.1.0
certifi                   2024.7.4
cffi                      1.17.0
charset-normalizer        3.4.1
click                     8.1.8
coverage                  7.8.0
cryptography              43.0.0
cxxheaderparser           1.4.1
debugpy                   1.8.11
delocate                  0.13.0
dill                      0.3.9
flexcache                 0.3
flexparser                0.4
h11                       0.14.0
httpcore                  1.0.5
httpx                     0.27.0
idna                      3.7
iniconfig                 2.0.0
invoke                    2.2.0
isort                     5.13.2
macholib                  1.16.3
mccabe                    0.7.0
mypy-extensions           1.0.0
numpy                     2.2.4
ollama                    0.3.1
opencv-python             4.11.0.86
packaging                 23.2
paramiko                  3.4.1
patch                     1.16
pathspec                  0.12.1
pcpp                      1.30
phoenix6                  24.3.0
photonlibpy               2025.2.1
Pint                      0.24.4
pip                       25.0.1
platformdirs              4.3.7
pluggy                    1.5.0
pybind11-stubgen          2.5.3
pycparser                 2.22
pydantic                  1.10.21
pyfrc                     2025.1.0
pylint                    3.0.3
PyNaCl                    1.5.0
pynetconsole              2.0.4
pyntcore                  2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/pyntcore
pytest                    8.3.2
pytest-reraise            2.1.2
PyYAML                    6.0.2
requests                  2.32.3
robotpy                   2025.3.2.0
robotpy-apriltag          2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-apriltag
robotpy-build             2025.1.0
robotpy-cli               2024.0.0
robotpy-cscore            2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-cscore
robotpy-hal               2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-hal
robotpy-halsim-ds-socket  2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-halsim-ds-socket
robotpy-halsim-gui        2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-halsim-gui
robotpy-halsim-ws         2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-halsim-ws
robotpy-installer         2025.1.0
robotpy-navx              2025.0.1.1
robotpy-playingwithfusion 2025.0.0
robotpy-rev               2025.0.3
robotpy-romi              2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-romi
robotpy-wpimath           2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-wpimath
robotpy-wpinet            2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-wpinet
robotpy-wpiutil           2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-wpiutil
robotpy-xrp               2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-xrp
setuptools                74.0.0
setuptools-scm            7.1.0
sniffio                   1.3.1
sphinxify                 0.12
tomli                     2.0.1
tomli_w                   1.2.0
tomlkit                   0.13.2
toposort                  1.10
typing_extensions         4.12.2
urllib3                   2.3.0
wheel                     0.45.1
wpilib                    2025.3.2.0                /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages
mikestitt@peas mostRobotPySpires2025 % robotpy sim                 
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/robotpy/main.py", line 245, in main
    cmd_class = entry_point.load()
                ^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/metadata/__init__.py", line 205, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/__init__.py", line 1, in <module>
    from . import _init_wpilib
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/_init_wpilib.py", line 14, in <module>
    import wpilib._impl._init_wpilibc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/_impl/__init__.py", line 1, in <module>
    from . import _init_wpilibc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/_impl/_init_wpilibc.py", line 15, in <module>
    _lib = CDLL(join(_root, "lib", "libwpilibc.dylib"), mode=RTLD_GLOBAL)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ctypes/__init__.py", line 379, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: dlopen(/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/_impl/lib/libwpilibc.dylib, 0x000A): Library not loaded: @loader_path/../../../ntcore/lib/libntcore.dylib
  Referenced from: <64235283-2B65-3D4F-868A-0BF932086DA9> /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/_impl/lib/libwpilibc.dylib
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/wpilib/_impl/lib/../../../ntcore/lib/libntcore.dylib' (no such file)
Plugin error detected in 'EntryPoint(name='run', value='wpilib._impl.start:Main', group='robotpy')' (use --ignore-plugin-errors to ignore this)
mikestitt@peas mostRobotPySpires2025 % 
mikestitt@peas mostRobotPySpires2025 % pip list                    
Package                   Version                   Editable project location
------------------------- ------------------------- -------------------------------------------------------------------------------
altgraph                  0.17.4
anyio                     4.4.0
appdirs                   1.4.4
astroid                   3.0.3
bcrypt                    4.2.0
black                     25.1.0
certifi                   2024.7.4
cffi                      1.17.0
charset-normalizer        3.4.1
click                     8.1.8
coverage                  7.8.0
cryptography              43.0.0
cxxheaderparser           1.4.1
debugpy                   1.8.11
delocate                  0.13.0
dill                      0.3.9
flexcache                 0.3
flexparser                0.4
h11                       0.14.0
httpcore                  1.0.5
httpx                     0.27.0
idna                      3.7
iniconfig                 2.0.0
invoke                    2.2.0
isort                     5.13.2
macholib                  1.16.3
mccabe                    0.7.0
mypy-extensions           1.0.0
numpy                     2.2.4
ollama                    0.3.1
opencv-python             4.11.0.86
packaging                 23.2
paramiko                  3.4.1
patch                     1.16
pathspec                  0.12.1
pcpp                      1.30
phoenix6                  24.3.0
photonlibpy               2025.2.1
Pint                      0.24.4
pip                       25.0.1
platformdirs              4.3.7
pluggy                    1.5.0
pybind11-stubgen          2.5.3
pycparser                 2.22
pydantic                  1.10.21
pyfrc                     2025.1.0
pylint                    3.0.3
PyNaCl                    1.5.0
pynetconsole              2.0.4
pyntcore                  2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/pyntcore
pytest                    8.3.2
pytest-reraise            2.1.2
PyYAML                    6.0.2
requests                  2.32.3
robotpy                   2025.3.2.0
robotpy-apriltag          2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-apriltag
robotpy-build             2025.1.0
robotpy-cli               2024.0.0
robotpy-cscore            2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-cscore
robotpy-hal               2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-hal
robotpy-halsim-ds-socket  2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-halsim-ds-socket
robotpy-halsim-gui        2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-halsim-gui
robotpy-halsim-ws         2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-halsim-ws
robotpy-installer         2025.1.0
robotpy-navx              2025.0.1.1
robotpy-playingwithfusion 2025.0.0
robotpy-rev               2025.0.3
robotpy-romi              2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-romi
robotpy-wpimath           2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-wpimath
robotpy-wpinet            2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-wpinet
robotpy-wpiutil           2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-wpiutil
robotpy-xrp               2025.3.2.1.dev4+g3f1e12ca /Users/mikestitt/mostrobotpy/subprojects/robotpy-xrp
setuptools                74.0.0
setuptools-scm            7.1.0
sniffio                   1.3.1
sphinxify                 0.12
tomli                     2.0.1
tomli_w                   1.2.0
tomlkit                   0.13.2
toposort                  1.10
typing_extensions         4.12.2
urllib3                   2.3.0
wheel                     0.45.1
wpilib                    2025.3.2.0                /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages

@virtuald
Copy link
Member

virtuald commented Apr 8, 2025

The error you're getting has to do with a quirk of how macOS support is implemented. What it comes down to is that you need to have all robotpy wrapper dependencies installed the same way (either via develop or installed normally). If you mix it then you sometimes get weird errors like that.

Typically, I use virtualenv (and specifically, virtualenvwrapper) to manage my various python development environments. I don't recommend that normal RobotPy users use virtualenvs unless they're comfortable with them, but they're really useful for python development -- and because they're effectively disposable, you can do a lot of weird changes in one environment without worrying too much about how much you're breaking. Specifically, I have different environments set up for installation from pypi, development, and various branches that I might be working on. If you were using a virtualenv, I would just say delete your virtualenv and start installing packages from scratch.

If you're not using a virtualenv, then just uninstall all robotpy-* packages and wpilib and pyntcore, and then run develop again.

@MikeStitt
Copy link
Author

I normally use python3 -m venv .venv, but I made a run without it using my macos python environment.

When I re-ran that in an empty .venv, I ended up with the same family of errors.

My conclusion is the robotpy-rev is not compatible with ./rdev.sh develop

More basic though is that https://github.com/robotpy/mostrobotpy/blob/main/README.md should define the relationship (what they do and where they are and how to find instructions to build/install them) between pc robotpy, the robotpy that get's create by mostrobotpy, which I assume is roborio robotpy, and mostrobotpy.

Also more basic is that https://github.com/robotpy/mostrobotpy/blob/main/README.md should define when to run:

python3 -m pip install robotpy relative to

pip install -r rdev_requirements.txt
pip install numpy

and

./rdev.sh ci run

and

./rdev.sh develop

and the command to python -m pip uninstall -r somefilename.txt to fix up python modules that are installed by python3 -m pip install robotpy but need to be removed before ./rdev.sh develop

then the installation instructions should take one all the way to a git clone of a robotpy project that supports a working:

robotpy sim

@virtuald
Copy link
Member

virtuald commented Apr 8, 2025

My conclusion is the robotpy-rev is not compatible with ./rdev.sh develop

Yes and no.

As mentioned above:

The error you're getting has to do with a quirk of how macOS support is implemented. What it comes down to is that you need to have all robotpy wrapper dependencies installed the same way (either via develop or installed normally). If you mix it then you sometimes get weird errors like that.

What this means is that if you are installing the base robotpy packages in develop mode, you must also install robotpy-rev, ctre, etc in develop mode. To install robotpy-rev, you would need to git clone the repo, and then run python setup.py develop. This only applies to macOS -- on Linux you would be able to pip install robotpy-rev and it would peacefully co-exist with your development install of the other robotpy components.

There might be a way out of the macOS problem by messing with the library name, but I haven't had time to look deeper into it.

I agree there should be more documentation. I also agree the installer scripts should be better to make this easier.

@MikeStitt
Copy link
Author

I have been trying lots of different paths to build mostrobotpy editable on OSX, Debian, and Windows, on a full production robot code base in simulation.

Goal is to be able to run

robotpy sim

on real robot code.
Generally what I have been doing is to:

(I'm typing not cut and pasting so there may be some typo's here)

python -m venv .venv
. .venv/bin/activate
python -m pip install robotpy
git clone mostrobotpy_URL
cd mostrobotpy
python -m pip install -r rdev_requirements.txt
python -m pip install numpy
python -m pip install robotpy_build

# repeatably pip uninstall the repos that were installed by robotpy that need to be uninstalled for a develop build to work until none are installed

#export the env variables to run ccache and multiple processors
./rdev.sh develop
cd ..
git clone robotpy-rev_URL
cd robotpy-rev
python setup.py develop

I end up with two copies of many of the repos needed in the system:

(.venv) mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable/robotpy-rev$ pip list
Package                  Version              Editable project location
------------------------ -------------------- --------------------------------------------------------------------------------------------------------------------------------------------------
black                    25.1.0
certifi                  2025.1.31
charset-normalizer       3.4.1
click                    8.1.8
cxxheaderparser          1.4.1
distro                   1.9.0
idna                     3.10
iniconfig                2.1.0
mypy-extensions          1.0.0
numpy                    2.2.4
packaging                24.2
patch                    1.16
pathspec                 0.12.1
pcpp                     1.30
pip                      23.0.1
platformdirs             4.3.7
pluggy                   1.5.0
pybind11-stubgen         2.5.3
pydantic                 1.10.21
pyntcore                 2025.3.2.0           /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/.venv/lib/python3.11/site-packages/pyntcore-2025.3.2.0-py3.11-linux-aarch64.egg
pyntcore                 0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/pyntcore
pytest                   8.3.5
PyYAML                   6.0.2
requests                 2.32.3
robotpy                  2025.3.2.0
robotpy-apriltag         0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-apriltag
robotpy-build            2025.1.0
robotpy-cli              2024.0.0
robotpy-cscore           0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-cscore
robotpy-hal              2025.3.2.0           /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/.venv/lib/python3.11/site-packages/robotpy_hal-2025.3.2.0-py3.11-linux-aarch64.egg
robotpy-hal              0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-hal
robotpy-halsim-ds-socket 0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-halsim-ds-socket
robotpy-halsim-gui       0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-halsim-gui
robotpy-halsim-ws        0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-halsim-ws
robotpy-rev              2025.0.3             /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/robotpy-rev
robotpy-romi             0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-romi
robotpy-wpilib-utilities 2025.0.0
robotpy-wpimath          2025.3.2.0           /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/.venv/lib/python3.11/site-packages/robotpy_wpimath-2025.3.2.0-py3.11-linux-aarch64.egg
robotpy-wpimath          0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-wpimath
robotpy-wpinet           2025.3.2.0           /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/.venv/lib/python3.11/site-packages/robotpy_wpinet-2025.3.2.0-py3.11-linux-aarch64.egg
robotpy-wpinet           0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-wpinet
robotpy-wpiutil          2025.3.2.0           /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/.venv/lib/python3.11/site-packages/robotpy_wpiutil-2025.3.2.0-py3.11-linux-aarch64.egg
robotpy-wpiutil          0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-wpiutil
robotpy-xrp              0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-xrp
setuptools               66.1.1
setuptools-scm           7.1.0
sphinxify                0.12
tomli                    2.2.1
tomli_w                  1.2.0
tomlkit                  0.13.2
toposort                 1.10
trpbe                    2025.0.1             /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable
typing_extensions        4.13.2
urllib3                  2.4.0
wheel                    0.45.1
wpilib                   2025.3.2.0           /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/.venv/lib/python3.11/site-packages/wpilib-2025.3.2.0-py3.11-linux-aarch64.egg
wpilib                   0.1.dev3180+g21bab63 /home/mikes/first/pythonExperiments/tryRobotPyBuildEditable/mostrobotpy/subprojects/robotpy-wpilib

I can uninstall those with pip unstinall repo==version commands.

But then I need to install a specific version of photonlibpy:

python -m pip install photonlibpy==version

which will then also uninstalls the editable development parts of robotpy

Note that I have been automating these steps here: https://github.com/MikeStitt/pythonExperiments/tree/main/tryRobotPyBuildEditable

But as I search online for ways to tell pip to install libraries without overwriting existing editable or development libraries, it seems like I might not be following normal development practices, and there are not easy ways to keep pip from overwriting editable development libaries as more tools get installed.

This seems to be the same on debian, and osx, (I haven't got my windows build to complete yet).

Thoughts on a valid development/debugging/testing installation process?

-Mike

@virtuald
Copy link
Member

virtuald commented Apr 15, 2025

The multiple package problem is likely caused by pip. I note that you have 23.1 installed, try upgrading.

The version numbers for your editable installs are 0.1.dev3180+g21bab63... which is really really weird. I would have expected them to start with the latest tag. This might be either pip, setuptools, or setuptools-scm causing a problem (or perhaps you don't have the full git history checked out?). I believe the version numbers for editable installs work fine for me at home, but I can't check now.

photonlibpy has a pinned dependency on >=2025.3.2, so when you pip install it, pip notices that the wrong version is installed (as noted above) and installs a newer one.

Telling pip to ignore dependencies (via --no-deps) is probably the only way to resolve this unless you can fix the version numbers on your editable installs. On the one hand, while this isn't recommended for general usage, in my opinion it's perfectly acceptable for a development environment where you're trying to control all the dependencies.

@virtuald
Copy link
Member

One last note, because I don't trust setuptools to do the right thing, I run python setup.py develop -N to make sure that it also doesn't try to install dependencies -- so I'd recommend that for your robotpy-rev install.

We will not be using setuptools next year, so hopefully some of that will go away.

@MikeStitt
Copy link
Author

My windows builds using x64 Native Tools Command Prompt fail this way:

"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:c:\users\mikestitt\documents\first\pythonexperiments\tryrobotpybuildeditable\mostrobotpy\subprojects\robotpy-wpiutil\wpiutil\lib /LIBPATH:c:\users\mikestitt\documents\first\pythonexperiments\tryrobotpybuildeditable\mostrobotpy\subprojects\robotpy-wpinet\wpinet\lib /LIBPATH:c:\users\mikestitt\documents\first\pythonexperiments\tryrobotpybuildeditable\mostrobotpy\subprojects\pyntcore\ntcore\lib /LIBPATH:build\temp.win-amd64-cpython-312\dlstatic\opencv_cpp\lib /LIBPATH:build\temp.win-amd64-cpython-312\dlstatic\cscore_cpp\lib /LIBPATH:build\temp.win-amd64-cpython-312\dlstatic\cameraserver_cpp\lib /LIBPATH:C:\Users\MikeStitt\Documents\first\pythonExperiments\tryRobotPyBuildEditable\.venvTrpbe\libs /LIBPATH:C:\Users\MikeStitt\AppData\Local\Programs\Python\Python312\libs /LIBPATH:C:\Users\MikeStitt\AppData\Local\Programs\Python\Python312 /LIBPATH:C:\Users\MikeStitt\Documents\first\pythonExperiments\tryRobotPyBuildEditable\.venvTrpbe\PCbuild\amd64 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22000.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22000.0\\um\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22000.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22000.0\\um\x64" cameraserver.lib cscore.lib opencv480.lib ntcore.lib wpinet.lib wpiutil.lib /EXPORT:PyInit__cscore build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\CameraServer.obj build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\cscore_cpp.obj build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\cscore_cv.obj build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\cscore_oo.obj build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\cscore_runloop.obj build\temp.win-amd64-cpython-312\Release\cscore\cvnp\cvnp.obj build\temp.win-amd64-cpython-312\Release\cscore\cvnp\cvnp_synonyms.obj build\temp.win-amd64-cpython-312\Release\cscore\src\main.obj /OUT:build\lib.win-amd64-cpython-312\cscore\_cscore.cp312-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\_cscore.cp312-win_amd64.lib
   Creating library build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\_cscore.cp312-win_amd64.lib and object build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\gensrc\cscore\_cscore.cp312-win_amd64.exp
cscore.lib(cscore_cpp.obj) : error LNK2001: unresolved external symbol _Cnd_timedwait_for_unchecked
cscore.lib(HttpCameraImpl.obj) : error LNK2001: unresolved external symbol _Cnd_timedwait_for_unchecked
cscore.lib(SourceImpl.obj) : error LNK2001: unresolved external symbol _Cnd_timedwait_for_unchecked
cscore.lib(Telemetry.obj) : error LNK2001: unresolved external symbol _Cnd_timedwait_for_unchecked
cscore.lib(RawSinkImpl.obj) : error LNK2001: unresolved external symbol _Thrd_sleep_for
cscore.lib(HttpCameraImpl.obj) : error LNK2001: unresolved external symbol _Thrd_sleep_for
cscore.lib(MjpegServerImpl.obj) : error LNK2001: unresolved external symbol _Thrd_sleep_for
cscore.lib(MjpegServerImpl.obj) : error LNK2001: unresolved external symbol __std_search_1
build\lib.win-amd64-cpython-312\cscore\_cscore.cp312-win_amd64.pyd : fatal error LNK1120: 3 unresolved externals
error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.36.32532\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120

@virtuald
Copy link
Member

That error seems really familiar, I think we ran into it last year? I don't use Windows at all these days.

I think you need to upgrade Visual Studio... according to this CD post you need 17.9 or later?

@MikeStitt
Copy link
Author

This feels like a silly question, really bad planning on my part.

But in debian bookworm I get:

mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ rm -rf .venv
mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ python -m venv .venv
mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ . .venv/bin/activate
(.venv) mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ robotpy --help
bash: robotpy: command not found
(.venv) mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ python -m pip cache purge
Files removed: 18
(.venv) mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ python -m pip install robotpy
Collecting robotpy
  Downloading robotpy-2025.3.2.0-py3-none-any.whl (2.2 kB)
Collecting robotpy-wpiutil==2025.3.2.0
  Downloading robotpy_wpiutil-2025.3.2.0-cp311-cp311-manylinux_2_36_aarch64.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 8.7 MB/s eta 0:00:00
Collecting robotpy-wpinet==2025.3.2.0
  Downloading robotpy_wpinet-2025.3.2.0-cp311-cp311-manylinux_2_36_aarch64.whl (748 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 749.0/749.0 kB 13.5 MB/s eta 0:00:00
Collecting robotpy-wpimath==2025.3.2.0
  Downloading robotpy_wpimath-2025.3.2.0-cp311-cp311-manylinux_2_36_aarch64.whl (6.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 17.9 MB/s eta 0:00:00
Collecting pyntcore==2025.3.2.0
  Downloading pyntcore-2025.3.2.0-cp311-cp311-manylinux_2_36_aarch64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 25.2 MB/s eta 0:00:00
Collecting robotpy-hal==2025.3.2.0
  Downloading robotpy_hal-2025.3.2.0-cp311-cp311-manylinux_2_36_aarch64.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 27.7 MB/s eta 0:00:00
Collecting wpilib==2025.3.2.0
  Downloading wpilib-2025.3.2.0-cp311-cp311-manylinux_2_36_aarch64.whl (5.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 30.4 MB/s eta 0:00:00
Collecting robotpy-wpilib-utilities<2026.0.0,>=2025.0.0
  Downloading robotpy_wpilib_utilities-2025.0.0-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.0/49.0 kB 17.8 MB/s eta 0:00:00
Collecting robotpy-cli<2026.0.0,>=2024.0.0
  Downloading robotpy_cli-2024.0.0-py3-none-any.whl (8.9 kB)
Installing collected packages: robotpy-wpiutil, robotpy-cli, robotpy-wpinet, robotpy-wpimath, robotpy-hal, pyntcore, wpilib, robotpy-wpilib-utilities, robotpy
Successfully installed pyntcore-2025.3.2.0 robotpy-2025.3.2.0 robotpy-cli-2024.0.0 robotpy-hal-2025.3.2.0 robotpy-wpilib-utilities-2025.0.0 robotpy-wpimath-2025.3.2.0 robotpy-wpinet-2025.3.2.0 robotpy-wpiutil-2025.3.2.0 wpilib-2025.3.2.0
(.venv) mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ robotpy --help
usage: robotpy [-h] [--main MAIN_FILE] [-v] [--ignore-plugin-errors] {run} ...

RobotPy CLI. See below for subcommands to accomplish various tasks for your robot project.

positional arguments:
  {run}                 subcommands
    run                 Executes the robot code using the currently installed HAL (this is probably not what you want unless you're on the
                        roboRIO)

options:
  -h, --help            show this help message and exit
  --main MAIN_FILE      The file that contains your main robot class
  -v, --verbose         Enable debug logging
  --ignore-plugin-errors
                        Ignore errors caused by RobotPy plugins (probably should fix or replace instead!)
(.venv) mikes@debian-12:~/first/pythonExperiments/tryRobotPyBuildEditable$ 


Debian bookworm doesn't support robotpy sim?

@virtuald
Copy link
Member

WPILib does not support simulation on linux/aarch64

@virtuald
Copy link
Member

Actually, that doesn't seem to be true anymore, https://frcmaven.wpi.edu/ui/native/release/edu/wpi/first/halsim/halsim_gui/2025.3.2/ shows aarch64 packages. We would need to update mostrobotpy to build them and update the robotpy-meta constraints.

@auscompgeek
Copy link
Member

auscompgeek commented Apr 17, 2025

I thought we do build it, it's just that by default robotpy-meta doesn't install the sim stuff on Linux aarch64?

@MikeStitt
Copy link
Author

MikeStitt commented Apr 18, 2025

I have found a route to simulate a full robot with mostrobotpy editable on a OSX M3 Mac:

First install ccache

Then:

git clone [email protected]:MikeStitt/pythonExperiments.git
cd pythonExperiments
cd tryRobotPyBuildEditable
deactivate ; rm -rf .venvTryRobotPyBuildEditable mostrobotpy mostRobotPySpires2025 robotpy-rev ; python3 -m venv .venvTryRobotPyBuildEditable ; . .venvTryRobotPyBuildEditable/bin/activate ; pip install -e . ; 

Python click module does most of the heavy lifting:

trpbe --quiet --toml trpbeConfigMikeStitt.toml doeditable

Which runs these commands:

command=git clone [email protected]:MikeStitt/mostrobotpy.git mostrobotpy
command=git -C mostrobotpy checkout pyIterativeAndTimedRobot
command=git clone https://github.com/robotpy/robotpy-rev robotpy-rev
command=git -C robotpy-rev checkout main
command=git clone [email protected]:spiresfrc9106/spiresRobot2025.git mostRobotPySpires2025
command=git -C mostRobotPySpires2025 checkout exp-timedrobotpy
command=pip install robotpy
command=cd mostrobotpy
command=pip install -r rdev_requirements.txt
command=pip install numpy
command=cd ..
command=pip install robotpy-build
command=cd mostRobotPySpires2025
command=python -m robotpy sync
command=cd ..
command=pip list --format json
command=pip uninstall -y robotpy-cscore
command=pip uninstall -y robotpy-wpinet
command=pip uninstall -y robotpy-apriltag
command=pip uninstall -y pyntcore
command=pip uninstall -y robotpy-wpiutil
command=pip uninstall -y robotpy-hal
command=pip uninstall -y robotpy-wpimath
command=pip uninstall -y wpilib
command=pip uninstall -y robotpy-halsim-gui
command=pip uninstall -y robotpy-rev
command=pip list --format json
command=cd mostrobotpy
command=python -m devtools develop
command=cd ..
command=cd robotpy-rev
command=python setup.py develop -N
command=cd ..

Then:

cd mostRobotPySpires2025
robotpy sim

Things that were not obvious were:

  • robotpy is a separate tool from mostrobotpy
  • Don't run /rdev.sh ci run because it builds and installs mostrobotpy and we don't want mostrobotpy installed when in development mode
  • If one doesn't run /rdev.sh ci run one needs to pip install robotpy-build in order for the build editable to work.
  • It's best to use robotpy sync to get all of the libaries synced for the robotpy sim before building mostrbootpy because robotpy sync and most pip installs will overwrite the editable mostrobotpy
  • That the libaries that are built editable need to be uninstalled prior to building them editable (this is a basic truth in python development)
  • That it's wise to repeatedly uninstall all the editable libraries until none are installed because sometimes one might have multilibraries installed with the same name
  • That robotpy-rev should be built: python setup.py develop -N with the -N to keep it from replacing other editable libraries.
  • That x64 linux and the intel Mac or M3 Mac are supported for building and simulating robotpy editable. x64 windows might be supported based upon a chief delphi post. linux/aarch64 is supported for building editable but not quite supported for simulation.

@auscompgeek
Copy link
Member

linux/aarch64 is supported for building editable but not quite supported for simulation.

Simulation should work fine on Linux aarch64. I've run the simulation for the examples on my Raspberry Pi 400.

@MikeStitt
Copy link
Author

linux/aarch64 is supported for building editable but not quite supported for simulation.

Simulation should work fine on Linux aarch64. I've run the simulation for the examples on my Raspberry Pi 400.

I created this issue to document where we are currently at: robotpy/robotpy-meta#33

Signed-off-by: Mike Stitt <[email protected]>
Signed-off-by: Mike Stitt <[email protected]>
Signed-off-by: Mike Stitt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants