Make the parser compile for Windows and Linux machines

[PolarGoose]

Description

I have tried the proposal from @matthijskooijman and managed to make the parser compile and work on Windows and Linux machines. I have only tested it on Windows, but there shouldn't be any problems with Linux.
This can be very beneficial because the parser can be used on Raspberry Pi and other Linux-based embedded boards, for example.

Content of the pull request

• Replace the usage of Arduino String with std::string
• Add a unit test project to test the parser

[LarsStegman]

This is great! When this is merged, ESPHome can be updated to use the new version. This should finally fix this issue: esphome/issues#3982

[PolarGoose]

@LarsStegman,
Nice, then there is even a real use case for this already.
From the issue that you have mentioned I see that the ESP Home uses the glmnet/Dsmr fork of this library. It means that we will also need to coordinate with the fork owner about merging these changes into his fork.
I would like to chat with you about it, could you please contact me to my [email protected]?

[glmnet]

Thanks! I will like this PR be tested in ESPHome before pushing a version

[LarsStegman]

@PolarGoose will you continue the work on this PR?

[PolarGoose]

I want to finish this pull request. However, I don't know much about ESPHome and how it can be tested.
@LarsStegman and @glmnet, can you test these changes on ESPHome after I finish them?

[LarsStegman]

I have no idea how to do that yet, but I can try.

[PolarGoose]

I have no idea how to do that yet, but I can try.

@glmnet, could you please help us with the testing when the pull request is finished?

[zuidwijk]

Hey @PolarGoose & @glmnet I'm more than willing to help and test. Please let me know how I can assist! I'm making esp32 based dsmr devices (but my programming/software skills arent good :) )

[PolarGoose]

@zuidwijk,

You make very interesting devices. Very cool.
So your use case is that you use this DSMR parser in ESP-home?

[zuidwijk]

@zuidwijk,

You make very interesting devices. Very cool.

So your use case is that you use this DSMR parser in ESP-home?

Yes, and it works when choosing type Arduino. But lately we get messages that it won't compile due to the size and we should use type esp-idf. But the code doesn't work with that.

[PolarGoose]

To add context:

• ESPHome DSMR implementation
  • It indeed depends on the Arduino framework
  • It only uses the P1Parser from this arduino-dsmr project. It implements the receiving state machine on its own.

This pull request already includes all the necessary code changes, and they have been tested via unit tests.
But, I haven't fixed the state machine that receives the packets. Luckily, this piece is not needed.

my programming/software skills arent good

@zuidwijk, you are actually a Code owner of ESPHome DSMR component :) and the implementation is pretty good.

What needs to be done:

• We need to merge this pull request to glmnet/Dsmr repo
• Create new PlatformIO package for the glmnet/Dsmr
• Make the ESPHome use new package
• The implementation of the ESPHome DSMR doesn't need to be changed.

@glmnet,
What do you think? We can either merge it into your repo, or we can create a separate repo and then make the ESPHome use it.

[zuidwijk]

To add context:

• ESPHome DSMR implementation

  • It indeed depends on the Arduino framework

  • It only uses the P1Parser from this arduino-dsmr project. It implements the receiving state machine on its own.

This pull request already includes all the necessary code changes, and they have been tested via unit tests.

But, I haven't fixed the state machine that receives the packets. Luckily, this piece is not needed.

my programming/software skills arent good

@zuidwijk, you are actually a Code owner of ESPHome DSMR component :) and the implementation is pretty good.

What needs to be done:

• We need to merge this pull request to glmnet/Dsmr repo

• Create new PlatformIO package for the glmnet/Dsmr

• Make the ESPHome use new package

• The implementation of the ESPHome DSMR doesn't need to be changed.

@glmnet,

What do you think? We can either merge it into your repo, or we can create a separate repo and then make the ESPHome use it.

Hey @PolarGoose, no I'm not the code owner, as in short: I've asked @glmnet to make the code and have paid him for that. Than later I've donated it to the community. Therefore I don't see me as owner 😁

@glmnet can you do this together with @PolarGoose ? I can help when testing is needed, got plenty of devices to test it on 😄

[matthijskooijman]

Maybe it's a bit off-topic, but just wanted to let you know that I'm still reading along with this and arduino-dsmr is still on my mind somewhere, but I just have too much other things on my mind to spend much time on this. If anything is needed on my side (which could be archiving this repository and redirecting to a fork of someone (or a group) that wants to take over maintenance, then I'm open to discuss that as well - maybe best in a separate issue).

[PolarGoose]

@zuidwijk, @matthijskooijman,

A bit off topic:
Do you have samples of DSMR packets from different smart meters? I think it would be worth open-sourcing. Otherwise, it is impossible for anyone to write or improve the parser.

[PolarGoose]

Update:
I'm working on adapting this pull request so that it can be merged into glmnet/Dsmr.

[PolarGoose]

Current progress:
https://github.com/PolarGoose/arduino-dsmr-make-work-on-Windows-and-Linux/tree/esphome_implementation
The code changes for the parser are done (I have described the details in the README)

I will look at how to integrate it into ESPHome. Shouldn't be a problem. I just never worked with ESPHome and very little with PlatformIO, which it uses.

[PolarGoose]

@zuidwijk,

I have finished the code changes.
This fork of Esphome compiles the DSMR component without Arduino:
https://github.com/PolarGoose/esphome-make-dsmr-compile-without-arduino
The implementation is unchanged. It gets the dependencies that don't depend on Arduino. It is enough. Thus, I think it should work the same way it worked before.

You can create an FW and test it..

If it works, we can create a pull request to esphome. You are the owner of the code. It should help with the pull request approval.

[zuidwijk]

I'll try it tomorrow. Otherwise after two weeks due to my vacation.

[PolarGoose]

@zuidwijk,

I have improved my fork of arduino-dsmr even further.
I think we can also simplify the ESPHome DSMR implementation.
Is it possible for you to send me an HW board? I could test it on my smart meter.
The only problem is if we change the code, how do we test it with Luxembourg's encrypted smart meters?

[PolarGoose]

@zuidwijk,

I have created a pull request to the ESPHome project:
esphome/esphome#11036
I have tested it on my "test bench". It works well. It receives unencrypted and encrypted telegrams.