Installer optimization to prevent blank page

[mr-manuel]

Hello Kevin,

what do you think about integrating a check, when installing GuiMods, to verify that the gui service started correctly after installation and if not then the modifications are reverted?

That would prevent a blank page if there is any error.

You could also use my overlay script to easily disable the modifications without reverting them: https://github.com/mr-manuel/venus-os_overlay-fs

You can check https://github.com/mr-manuel/venus-os_dbus-serialbattery/blob/master/dbus-serialbattery/enable.sh and https://github.com/mr-manuel/venus-os_dbus-serialbattery/blob/master/dbus-serialbattery/custom-gui-install.sh on how to install and use it.

Let me know, if you have questions.

[kwindrem]

Yes I did think about this but never implemented it. PackageManager runs constantly and could provide that mechanism. One complication is to determine which package is causing the gui app to crash. That would not be easy. So I may need to uninstall ALL packages.

[mpvader]

Hey both, a few alternative options:

• Only install all packages in case venus os version is in list of supported versions
• same, but then for gui version
• don’t replace files but patch them using the patch command line tool, and monitor exit code of that and abort when one fails

[kwindrem]

I just released a new version of SetupHelper that patches the import QtQuick line in all files to the proper version. This should resolve the current issue (white screen). When I tested this, I needed to uninstall then reinstall other packages as well. SetupHelper does provide a mechanism to blindly uninstall all packages should that need arise.

This may require reinstalling the firmware using the command line or the New UI.

I do support the notion of first and last firmware versions but currently don't set these unless I know the package becomes obsolete outside the supported versions. Setting an obsolete version to the last tested version would prevent a package installation when it would otherwise work fine and require an update to the package with all firmware updates.

I have been migrating my packages to patch vs replace but this does not work for files that get added to the system That is, the file doesn't exist in the stock firmware. Generally this is for added menus. A patch is checked for errors and the install is not allowed to modify files (or uninstall the package) if an error occurs.

I need to wait until the v3.54 beta ends before I can update GuiMods for v3.60~18 and beyond.

[kwindrem]

I'm in the process of testing a mechanism that checks the current firmware version against a list of "approved" firmware versions. It is working but Sebastian at 123electric raises a valid point: Doing this means that a package would fail to install on a new firmware version until the author "approved" that version and issued an update to the package. There are a lot of third party modifications that don't touch the gui. More devastating is that SetupHelper would uninstall itself, shutting down PackageManager. This would require a new blind install (or command line install) of SetupHelper.

So this approach isn't going to be practical. Venus OS is always in a state of testing with typically many beta versions released in a short period of time.

I looked for a way to determine what GUI version is in use but there is no GUI version file and its log does not report its version when it starts. (Adding that would not help supporting older firmware unfortunately.)

PackageManager relies on the Classic UI and is therefore subject to being uninstalled thus preventing package management of any kind.

Users can turn off automatic downloads and updates. If they do disable auto updates, PackageManager would not be able to install a new version that provides a fix even if it was available.

The GUI white screen does need to be addressed somehow because recovery requires tools often beyond a user's capability, especially for a remotely controlled system.

I looked at mr-manuel's overlay but in this case, I'd need to overlay the entire QML directory, copy the stock files from prior to the overlay then modify them in the overlay. This gets really complicated when considering multiple modifications from different authors.

If PackageManager could be made independent of the GUI then this whole problem would be much easier to handle since package management would still be available even if the GUI white screened. An HTML page comes to mind. That way anyone with a web browser could connect to venus.local/PackageManager and manage packages from there. I would probably remove the related GUI pages! I don't have the knowledge to do this however. This could also possibly be a node-red thing. Again my knowledge doesn't stretch that far.

Comments?

[dsrawding]

Since Venus uses nginx as the browser engine it is simple to add another virtual host even on a different port for PackageManager. All you have to do is drop a couple line config file in /etc/nginx/sites-available/ directory and restart the process. I could whip up a bash script to do that pretty quickly. Just need to determine where the new PackageManager pages would be located.

It probably would not be that difficult to incorporate into Node-Red as well. You can import a flow from an install script or just have the user import or cut/paste the flow from within Node-Red. This may be a good way as it's unlikely that it would be broken by a Victron update.

I'm not a Node-Red expert but do have many 100's of hours of experience with it in a number of applications. I'd be happy to lend a hand wherever I could.

[mr-manuel]

I'm in the process of testing a mechanism that checks the current firmware version against a list of "approved" firmware versions.

You don't need that for all packages, but only for packages that modify the gui or critical system files when starting up. I'm still convinced that the best option in this case would be, that every package specifies only the service that need to be restarted after installation. Then the SetupHelper restarts that service and checks it's function after some seconds. If the service don't run as expected, then the uninstall script is triggered and the Venus OS version, where the problem occured is saved to a file.

How to know which packet caused the issue? This should be really easy, since you are not installing all packages at once after an upgrade. One package is installed after the other and if one failed, then uninstall it.

Another option would be, that instead of an uninstall you trigger a reboot to load the backup firmware (downgrade) and then display a message to the user, that the new version rised errors with SetupHelper and the Venus OS was restored. In this case you need also to disable automatical updates to avoid a loop. This way the user don't need to install SetupHelper manually again.

Currently we are adding a log message in GUIv1, so that you can filter by that to know, if the GUI started successfully or not. I will provide more details here once implemented.

The GUI white screen does need to be addressed somehow because recovery requires tools often beyond a user's capability, especially for a remotely controlled system.

Not anymore, since you always can access GUIv2 via browser and gain SSH access. Therefore making a custom webpage for this would be overkill, since you always can access GUIv2.

I looked at mr-manuel's overlay but in this case, I'd need to overlay the entire QML directory, copy the stock files from prior to the overlay then modify them in the overlay. This gets really complicated when considering multiple modifications from different authors.

Why do you need to copy over the stock files? That is not how a overlay fs works. You only enable the overlay before you do the modifications. Everything else remains at it is. You overlay /opt/victronenergy/gui and then you still see all content of the folder like before with the only difference, that all modifications are saved in the overlay. There is no need to copy over files.

Best would be if you try it, then you see how it works. Install: https://github.com/mr-manuel/venus-os_overlay-fs#install--update
Then run bash /data/apps/overlay-fs/add-app-and-directory.sh GuiMods /opt/victronenergy/gui to enable the overlay. If you now browse the folder you see that everything is there and you even don't need to make the root-fs writable. If you change somethin in the overlayed folder it's saved in /data/apps/overlay-fs/data/gui/upper.

root@einstein:~# ls -l /opt/victronenergy/gui
-rwxr-xr-x    1 root     root        705116 Jan 17 10:44 gui
drwxr-xr-x    2 root     root          4096 Jan 17 10:44 lang
drwxr-xr-x    1 root     root          4096 Dec 29 20:41 qml
drwxr-xr-x    3 root     root          4096 Jan 17 10:44 qt-components
-rw-r--r--    1 root     root        208094 Jan 17 10:44 splash.png
-rwxr-xr-x    1 root     root           820 Jan 17 10:44 start-gui.sh
root@einstein:~#
root@einstein:~# ls -l /data/apps/overlay-fs/data/gui/upper/
drwxr-xr-x    2 root     root          4096 Dec 29 20:41 qml

[kwindrem]

Thanks for the comments.

Right now, PackageManager installs all packages needing an update before rebooting the system or restarting the GUI service. All other services are restarted before the setup script exits. I did this to avoid multiple GUI restarts if more than one package needed updating for example when a firmware update occurs. Changing this behavior and adding checks for proper service behavior will delay installs but may be the right approach. I figure it will take 5-10 seconds per install.

You don't need that for all packages, but only for packages that modify the gui or critical system files when starting up. I'm still convinced that the best option in this case would be, that every package specifies only the service that need to be restarted after installation. Then the SetupHelper restarts that service and checks its function after some seconds. If the service don't run as expected, then the uninstall script is triggered and the Venus OS version, where the problem occured is saved to a file.

Would this not be most packages? Seems that a package is modifying the GUI, updating a service or adding one. I'm looking for alternatives to the "approved firmware list" concept.

Overlay:

So on the overlay, you are saying that all files from the read-only partition will appear without needing to copy? That makes things much easier.

What happens if multiple packages overlay the same directory?

Does the overlay need to be enabled on every boot?

Currently we are adding a log message in GUIv1, so that you can filter by that to know, if the GUI started successfully or not. I will provide more details here once implemented.

This will help going forward but won't help supporting older firmware.

Not anymore, since you always can access GUIv2 via browser and gain SSH access. Therefore making a custom webpage for this would be overkill, since you always can access GUIv2.

Good to know. I'll add this to the recovery procedure.

[mr-manuel]

So on the overlay, you are saying that all files from the read-only partition will appear without needing to copy? That makes things much easier.

Yes

So on the overlay, you are saying that all files from the read-only partition will appear without needing to copy? That makes things much easier.
What happens if multiple packages overlay the same directory?

If you are using my script, you can overlay a folder only once. The overlay is used instead of making the root-fs read/write. This means everything else remains the same. You just need to add an overlay once during installation.

This will help going forward but won't help supporting older firmware.

Yes, but older firmware is already covered by you and all changes are known. Anyway I would not recommend to support too much firmware versions, but you changed that already to the latest 3, if I remeber correctly.

[mr-manuel]

With Venus OS v3.60~22 you will now see a log entry in /var/log/start-gui (Cerbo GX, Ekrano GX) or /var/log/gui (Raspberry Pi, GX card, ...) with

loading QML files failed

or

loading QML files succeeded

Working example:

2025-02-04 07:03:27.042478500 *** Starting gui ***
2025-02-04 07:03:27.263003500 using non virtual framebuffer fb0
2025-02-04 07:03:28.582734500 Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
2025-02-04 07:03:28.582745500 Qt depends on a UTF-8 locale, but has failed to switch to one.
2025-02-04 07:03:28.582747500 If this causes problems, reconfigure your locale. See the locale(1) manual
2025-02-04 07:03:28.582751500 for more information.
2025-02-04 07:03:28.864866500 "using /etc/venus for runtime features"
2025-02-04 07:03:28.869492500 "running on einstein"
2025-02-04 07:03:29.491380500 Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
2025-02-04 07:03:29.611893500 Creating settings
2025-02-04 07:03:30.288071500 [VeQItemExportedDbusServices] Registered service "debug.victronenergy.gui"
2025-02-04 07:03:31.339029500 language is "en"
2025-02-04 07:03:31.411738500 accessLevelChanged:  2
2025-02-04 07:03:31.417916500 setAccessLevel:  2
2025-02-04 07:03:31.612971500 LocalThemeDaemonClient: Looking for assets in "/opt/victronenergy/themes/ccgx"
2025-02-04 07:03:31.644093500 Screen: 800x480 110x60mm 184.727x203.2dpi
2025-02-04 07:03:31.824163500 TimeZone::valueChanged:  "/UTC"
2025-02-04 07:03:31.829608500 ScreenSaver::dbusDisplayOffTimeChanged:  600
2025-02-04 07:03:31.838486500 ScreenSaver::enable
2025-02-04 07:03:31.838809500 ScreenSaver::screenOn
2025-02-04 07:03:54.643039500 file:///opt/victronenergy/gui/qml/SignalToaster.qml:6:2: QML Connections: Implicitly defined onFoo properties in Connections are deprecated. Use this syntax instead: function onFoo(<arguments>) { ... }
2025-02-04 07:03:54.647160500 file:///opt/victronenergy/gui/qml/Toast.qml:38:2: QML Connections: Implicitly defined onFoo properties in Connections are deprecated. Use this syntax instead: function onFoo(<arguments>) { ... }
2025-02-04 07:03:54.648044500 file:///opt/victronenergy/gui/qml/main.qml:130:2: QML Connections: Implicitly defined onFoo properties in Connections are deprecated. Use this syntax instead: function onFoo(<arguments>) { ... }
2025-02-04 07:03:54.648360500 file:///opt/victronenergy/gui/qml/main.qml:121:2: QML Connections: Implicitly defined onFoo properties in Connections are deprecated. Use this syntax instead: function onFoo(<arguments>) { ... }
2025-02-04 07:03:54.896858500 file:///opt/victronenergy/gui/qml/PageMain.qml:250:2: QML Connections: Implicitly defined onFoo properties in Connections are deprecated. Use this syntax instead: function onFoo(<arguments>) { ... }
2025-02-04 07:03:55.161876500 loading QML files succeeded

Non working example:

2025-02-04 07:05:58.204878500 *** Starting gui ***
2025-02-04 07:05:58.283402500 using non virtual framebuffer fb0
2025-02-04 07:05:58.659311500 Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
2025-02-04 07:05:58.659320500 Qt depends on a UTF-8 locale, but has failed to switch to one.
2025-02-04 07:05:58.659323500 If this causes problems, reconfigure your locale. See the locale(1) manual
2025-02-04 07:05:58.659327500 for more information.
2025-02-04 07:05:58.744363500 "using /etc/venus for runtime features"
2025-02-04 07:05:58.752139500 "running on einstein"
2025-02-04 07:05:58.909724500 Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
2025-02-04 07:05:59.185193500 Creating settings
2025-02-04 07:05:59.518442500 [VeQItemExportedDbusServices] Registered service "debug.victronenergy.gui"
2025-02-04 07:06:00.078096500 language is "en"
2025-02-04 07:06:00.092025500 accessLevelChanged:  2
2025-02-04 07:06:00.092182500 setAccessLevel:  2
2025-02-04 07:06:00.211132500 LocalThemeDaemonClient: Looking for assets in "/opt/victronenergy/themes/ccgx"
2025-02-04 07:06:00.211713500 Screen: 800x480 110x60mm 184.727x203.2dpi
2025-02-04 07:06:00.281071500 TimeZone::valueChanged:  "/UTC"
2025-02-04 07:06:00.281687500 ScreenSaver::dbusDisplayOffTimeChanged:  600
2025-02-04 07:06:00.281988500 ScreenSaver::enable
2025-02-04 07:06:00.282064500 ScreenSaver::screenOn
2025-02-04 07:06:03.878686500 file:///opt/victronenergy/gui/qml/main.qml:12:15: Type PageMain unavailable
2025-02-04 07:06:03.878697500           initialPage: PageMain {}
2025-02-04 07:06:03.878699500                        ^
2025-02-04 07:06:03.879605500 file:///opt/victronenergy/gui/qml/PageMain.qml:40:26: Type PageSettings unavailable
2025-02-04 07:06:03.879612500                                   subpage: Component { PageSettings {} }
2025-02-04 07:06:03.879615500                                                        ^
2025-02-04 07:06:03.880243500 file:///opt/victronenergy/gui/qml/PageSettings.qml:50:5: Type PageSettingsSystem unavailable
2025-02-04 07:06:03.880250500                                   PageSettingsSystem {
2025-02-04 07:06:03.880252500                                   ^
2025-02-04 07:06:03.880856500 file:///opt/victronenergy/gui/qml/PageSettingsSystem.qml:157:5: Type PageSettingsBatteries unavailable
2025-02-04 07:06:03.880862500                                   PageSettingsBatteries {
2025-02-04 07:06:03.880865500                                   ^
2025-02-04 07:06:03.881375500 file:///opt/victronenergy/gui/qml/PageSettingsBatteries.qml:1:1: module "QtQuick" version 1.1 is not installed
2025-02-04 07:06:03.881383500      import QtQuick 1.1
2025-02-04 07:06:03.881385500      ^
2025-02-04 07:06:03.881538500 loading QML files failed

You can now check for that log lines and revert the installation or trigger a boot of the backup firmware. In case of a Setup Helper install failure it would be probably better to restart and load the old firmware. Else the user has to install Setup Helper again.

In that case we need to add a note somewhere, with which Venus OS version the installation failed, to prevent another firmware update again. Also automatic firmware update has to be disabled in case of failure, to prevent a loop. If it's ok for you, I open a PR for that.

I tested this code, to check, if the GUIv1 started successfully or not:

# Check if GUIv2 is running, if yes switch to GUIv1 temporarily
gui_version=$(dbus -y com.victronenergy.platform /Gui/RunningVersion GetValue)
if [[ "$gui_version" == *"value = 2"* ]]; then
    echo "GUI version 2 is running. Switching temporarily to GUI version 1 to verify changes."
    dbus -y com.victronenergy.settings /Settings/Gui/RunningVersion SetValue 1
    sleep 10
else
    echo "GUI version 1 is running"
fi

# Check log path
if [ -f /var/log/start-gui/current ]; then
    log=/var/log/start-gui/current
else
    log=/var/log/gui/current
fi

# Create variable to store if GUI started successfully
gui_started_successfully=0

# Wait until specific string is found in log, not older than 60 seconds
while true; do
    current_time=$(date +%s)
    time_60_seconds_ago=$((current_time - 60))
    formatted_time=$(date -d "@$time_60_seconds_ago" '+%Y-%m-%d %H:%M:%S')

    if tail $log | tai64nlocal | awk -v d="$formatted_time" '$1" "$2 >= d {print $0}' | grep -q "loading QML files succeeded"; then
        gui_started_successfully=1
        break
    elif tail $log | tai64nlocal | awk -v d="$formatted_time" '$1" "$2 >= d {print $0}' | grep -q "loading QML files failed"; then
        break
    fi
    sleep 1
done

# Revert to GUIv2, if GUIv2 was running
if [[ "$gui_version" == *"value = 2"* ]]; then
    echo "Switching back to GUI version 2"
    dbus -y com.victronenergy.settings /Settings/Gui/RunningVersion SetValue 2
fi

# Elaborate result
if [ $gui_started_successfully -eq 1 ]; then
    echo "GUIv1 started successfully"
else
    echo "GUIv1 failed to start, revert changes/boot backup firmware"
fi