Updates stopping, unnecessary creation of new entities

[mmccool]

I use this to track battery state-of-charge state on various Apple devices to control chargers so that I don’t overcharge my Apple devices (keeping charge below 80% is my target). Unfortunately only the latest iPhone has this feature built in, because Apple.

Anyway, this does not quite work well for two reasons:

1. Sometimes the devices just stop updating and the automations then use stale state. Restarting the service sometimes fixes it but, see 2. When devices stop updating it does not necessarily affect all the devices. We do have a lot of devices (6, plus a Watch) and am running complications on the Watch. API limits?
2. Sometimes when the service restarts it creates new entities for all the sensors, which then breaks all the automations and dashboards.

Due to the “sometimes” in 2 it may happen on an update. Not sure when the last update was but it last happened 1 hour ago after I tried to reload the service (but not HA) to try and fix an instance of 1.

[gcobb321]

1. Doing a reload will stop the current instance of iCloud3 and load the code all over again. HA will create the _2 extension on the entity name since iCloud3 was running and the sensor entities exist in HA’s memory tables. On the reload, HA sees that they entities exist and it adds the _2 to create unique entities instead of reusing the ones iCloud3 wants.

What you want to do is an Event Log > Actions > Restart iCloud3. ICloud3 will then restart and it will initialize itself and refresh all locations.

2. iCloud3 updates the battery info from the mobile app or the Apple account data, using the latest value. It is displayed in the Event Log when it is changed. Look there when you think it has stopped updating and see if old data was being read or if there was some other activity. Set Log Level to debug (Configure Settings > Menu Page 2 > Format Parameters screen) which will add more info to ‘icloud3-0.log’ file which might show what is or is not happening when this occurs.

[mmccool]

OK, the reload vs restart explains 2. Still… this does not seem like the right behavior. Other integrations don’t have this problem.

As for 1 - I will try to capture some logs to see better what is going on.

[JtwoA]

Other integrations don’t have this problem.

sigh

[gcobb321]

@JtwoA
I have been able to recreate it on the device tracker by having a device (gary_iphone), deleting it in Configure Settings > Update Devices and then reading the same device (c creates gary\iphone_2). HA has a deleted devices list in the device registry. I think then it readds it, it thinks it still exists.

[gcobb321]

@JtwoA
I spent most of yesterday trying to solve this and haven't found the solution. HA is assigning the _2 when it finds the entity in memory or in the deleted entities list. I tried all kinds of things to get HA to use the existing one or to create it without the _2. Could you let me give me the name of the other integrations that do not have this problem so I can look at their code. i agree with the sigh but the HA docs are limited, I am one guy, do not staff of programmers that know what they are doing and have to figure out everything as I go along. BIG SIGH

[JtwoA]

@JtwoA I spent most of yesterday trying to solve this and haven't found the solution. HA is assigning the _2 when it finds the entity in memory or in the deleted entities list. I tried all kinds of things to get HA to use the existing one or to create it without the _2. Could you let me give me the name of the other integrations that do not have this problem so I can look at their code. i agree with the sigh but the HA docs are limited, I am one guy, do not staff of programmers that know what they are doing and have to figure out everything as I go along. BIG SIGH

My sigh was directed at the poster who fussed about "other integrations not having this issue" as though you're a paid dev who is doing this full-time. I personally think you've done a fantastic job dealing with Apple's recent changes (as someone who deals with them for business, I understand COMPLETELY).

[christophertjones123]

@gcobb321 i noticed no one ever gave you an example integration that doesn't have this problem so i thought i would provide it to you. I currently use the alarm.com integration and it does not have this problem on reload.

https://github.com/pyalarmdotcom/alarmdotcom

[gcobb321]

Thanks,

As it turns out, I completed changes to iC3 several days ago to detect and correct the sensor entity_id _2 error during startup or a reload. An item was added to Configure > Tools in case something slips by the startup checks or to correct it if does reoccur.

This will be in v3.1.4.2

[christophertjones123]

@gcobb321 excellent, thanks again for all the work on this. its a great integration

[gcobb321]

@JtwoA @christophertjones123
Here is a sneak peak at the v3.1.4.2 update

Unzip into the \icloud3 directory and restart HA
icloud3 v3.1.4.2.zip

Release Notes - v3.1.4.2 (1/12/2024)

General Updates

1. HACS (Updated) - Changed the method of launching the routine that checks HACS to see if there is an iCloud3 Update so HA won't complain.
2. INTERNET CONNECTION ERROR (Updated) - Improved the handling of problems when the Internet is not available. If an error occurs when iCloud3 is starting, tasks related to connecting to Apple servers are bypassed. All device tracking will be Paused for 20-minutes. iCloud3 will restart ot tracking will resume to see if an internet connection can be established. If the internet is still down, tracking will be paused for another 20-minutes. The 'Home Assistant is Offline' messsage will be displayed in the HA Error Logs and as an alert on the Event Log screen.
3. WAZE HISTORY DAILY MAINTENANCE (Update) - The Waze History Database end-of-day process includes updating the Waze Map Tracking sensor and other activities to tune the data base. Various activity messages were displayed on the first devices 'info' sensor. HA could not keep up with adding these update to the History Database. This was fixed by only displaying a progress counter to the Event Log instead.
4. OTHER - Various message formatting improvements and other stuff I can not remember.

Configuration Screen Changes

1. APPLE ACCOUNT LOGIN (Fixed)
   • Fixed a problem verifying the password before logging into the Apple account when iCloud3 was starting.
   • Fixed a problem where a valid password may fail the initial verification test and an invalid password may pass the validation test.
   • Fixed a problem where a password that failed the initial verification test could not be corrected on Configure > Data Sources ... > Apple Username/Password screen.
2. UPDATE ICLOUD3 DEVICE SCREEN (Fixed) - Fixed a problem where changing the Apple account and/or Mobile App device selection would be saved in the configuration file but not used when iCloud3 was restarted after the Configure update.
3. SENSORS SCREEN (Updated) - Added an Action option to reset the sensor seletion values to their initial configuration.
4. OTHER TRACKING PARAMETERS SCREEN (Fix) - Fixed an error when saving changes on this screen.
5. TOOLS (New Items) - Added HA Restart/Reload iCloud3 and _Correct 2 Sensor Name Error correction tools.

[christophertjones123]

@gcobb321

I tested this out, and this does in fact fix this particular issue. I had several entities that existed as "Not Provided" and on reload of the integration it corrected it. I can also confirm that I have manually reloaded the integration to try and replicate the issue and can no longer replicate it.

Excellent work. Thank you, this is great. This issue never really bugged me except for organizational purposes, but having it resolved does help my brain not lose it.

[gcobb321]

That issue has made my brain hurt for a while. I spent about a week on it in Dec but could never figure out what HA was doing. Then looked at some code from the Places app and could finally see what I needed to do.

Glad things are working out. I did find some Debug code that I missed dealing with lost internet issues. Final testing tomorrow and will update the developers GitHub repository. Then the general on Sunday.