OpenDSS version

[mackei]

Hi,

not really an issue, more of a documentation question. Where can we find which OpenDSS version is dss_sharp built against?

Regards,
Maciej

[PMeira]

The usual version string in DSS.Version contains most of the info:

DSS C-API Library version 0.12.1 revision 7230c9ec1f69ee2f13ac11202ced49ab0b239ebb based on OpenDSS SVN 3460 [FPC 3.2.2] (64-bit build) MVMULT INCREMENTAL_Y CONTEXT_API PM 20220716135448; License Status: Open

From the C-API version, you can see more in the changelog at https://github.com/dss-extensions/dss_capi/blob/master/docs/changelog.md#version-0121 and most of the omissions from our fork at listed at https://github.com/dss-extensions/dss_capi/blob/master/docs/known_differences.md

Revision 7230c9ec1f69ee2f13ac11202ced49ab0b239ebb is the Git commit hash from the DSS C-API repo.

SVN 3460 can be seem through the official repo at https://sourceforge.net/p/electricdss/code/HEAD/tree/trunk/
So, a few commits before the latest OpenDSS release.

[FPC 3.2.2] is the compiler, and the following are flags for some of our extensions:

• MVMULT: a compile flag to use our matrix multiplication using Eigen in C++ instead of the original Pascal version
• INCREMENTAL_Y: the support for incremental Y updates was compiled (still disabled by default)
• CONTEXT_API: the support for multiple DSS contexts is enabled
• PM: the support for the official parallel-machine mechanism based on actors is built-in.

And 20220716135448 is the timestamp when the binary was built.

License Status: Open is kept from the official OpenDSS version string, which is something like:

Version 9.4.2.2 (64-bit build); License Status: Open

[mackei]

Thanks. In my case, I wanted to cross-verify my dss_sharp program with an OpenDSS script, so I wanted to install standalone OpenDSS in a version that'd match dss_sharp build. With the above infomation, I could check in the OpenDSS history for the SVN 3460 changeset, and the 0.12.1 version of DSS C-API is somewhere between OpenDSS Version 9.4.1.2 and Version 9.4.2.1.
Any thoughts on including the OpenDSS Version somewhere in there more explicitly?

[PMeira]

I don't think it's that simple. Listing a version can lead to wrong conclusions since most users don't follow closely the development of either the official version nor DSS C-API.

The codebase is quite different and we port things selectively after testing and cross-validating ourselves. Technically we would tag our latest release to match OpenDSS v9.4.2.1 for example, but the official version had a bug that affected a lot of systems. We noticed something was off and reverted the change for our release. If we mentioned it was based on v9.4.2.1, the results would be off.

Besides occasional bugs, not every commit there needs to be ported for various reasons, be it code for features not exposed in the official OpenDSS (e.g. a lot of commits go to OpenDSSCmd for FPC, which is not exactly official), GUI, documentation, or just due to the different code architecture of DSS C-API v0.12.x.

Maybe you found this: nowadays there is a .txt in the official repo for the version at https://sourceforge.net/p/electricdss/code/HEAD/tree/trunk/Version8/Source/Current_ver.txt -- replace HEAD with the SVN revision and we usually should be between that and the previous version. I'd recommend keeping a known valid set of results to validate the new OpenDSS versions so you don't assume there are issues here. Most of the OpenDSS components are very stable and should not change from one release to another -- if something does and it's not explained in the changelog, it's usually a bug.

[PMeira]

I'll see if I can automate grabbing the version from that txt. If that works fine, as long as part of my previous comments are left in the DSS.Version comments/docstring, that seems good enough for the current situation.

[mackei]

That's definitely not very important, just a cosmetic thing - as you explained, there's ways to get at that version. But thanks!
If I may, however, I think I'd recommend trying to build DSS C-API and DSS Sharp against official versions of OpenDSS (as much as possible with the selective porting of features). If nothing else, that'd just be clearer - this version is built off XXX OpenDSS version. Like you said, the components should be stable at this point, so not taking the absolutely latest OpenDSS changes shouldn't be a big deal. If latest OpenDSS version has a known issue - just take a previous, but official stable version. Again, the components should be stable at this point, so downgrading one minor version shouldn't matter. And in an off chance that there is another bug (not known yet) which found its way into either OpenDSS or DSS Sharp, it's much easier to compare the results (and be confident in them) if you know for sure that they are using the same version of the key solver.

[PMeira]

Text wall warning! TLDR: There are a few ways things can progress, only time will tell, but right now DSS C-API cannot use the original/official OpenDSS code anymore.

If I may, however, I think I'd recommend trying to build DSS C-API and DSS Sharp against official versions of OpenDSS (as much as possible with the selective porting of features).

That can't be done anymore. The only feature we could add without much hassle would be mapping OpenDSS errors to exceptions, everything else including the API performance cannot be achieved.

We've gone through a few phases already:

1. In 2016-2017, we tried to use the official OpenDSSDirect.DLL. Besides the bugs and some limitations, in the end we had the same issues as the official COM OpenDSSengine.DLL. That's when we started developing and testing DSS C-API and DSS Python.

2. In 2018, we felt it was mature enough. The first public releases of DSS C-API were basically what you proposed -- use the official version with minimal changes. It was maintained as a set of patches in hopes that EPRI would integrate it someday. That's why https://github.com/dss-extensions/electricdss-src existed. You can follow some of the conversations on https://sourceforge.net/p/electricdss/discussion/861976/thread/525c13df/ and https://sourceforge.net/p/electricdss/discussion/861977/thread/89f78cd6/?limit=25#2d06

3. In 2019, a year or so after the first public release, that become impractical as we added more and more fixes. Since it was clear a direct contribution wouldn't work, we started addressing other issues in the API code and DSS C-API became more of a friendly fork than a set of patches.

4. By 2020, we already had some internal branches at Unicamp testing some of the bonus features we released in 2022. There are still some extra features we didn't publish.

The DSS C-API library nowadays:

• Probably cannot be built with Delphi without some changes, so Free Pascal is a requirement. This is in direct conflict with the official implementation, which only supports Delphi. OpenDSSCmd is built with Free Pascal and, since it lacks the changes from DSS C-API, has a lot of bugs and limitations. I already expressed my concerns in https://sourceforge.net/p/electricdss/discussion/861976/thread/7172875585/
• Uses a fork of KLUSolve with extra features that allow better performance and incremental Y updates.
• Has a rewritten IO system replacing the limited and deprecated Pascal IO system. This is what enabled using ZIP files as input.
• Supports UTF8 and is ready for i18n. Even if ASCII is fine for a lot of users, I'm in Brazil, so this is important in the long term.
• Supports multiple DSS engines. This required changing a lot of code to replace global variables, so cannot be done with the current OpenDSS version.
• Complex numbers use operator overloads, resulting in cleaner code.
• The whole property system was rewritten. This removed a whole class of bugs from the implementation -- some recent OpenDSS bugs fall on this and they don't affect us. This also changed a lot of files in the codebase. This allows exposing every single data class from OpenDSS -- currently we do it in dss.hpp and DSS Python, see for example https://dss-extensions.org/dss_python/obj.html
• The library and bindings are built and tested on most the most platforms and processor architectures. The official version is restricted to Windows x86 and x64.

I'm working on finishing dss-extensions/dss_capi#88 next week. After that and reintegrating A-Diakoptics, DSS C-API v1.0 is almost ready. DSS Sharp itself needs some good examples and maybe a COM layer (so it can be used as a COM module) to reach v1.0.

If you look in the DSS C-API repo, you will see that nearly all of the changes in the last couple of years are made by myself, with some ports from the SVN. Even the official repository is very restricted in terms of contributors. This is terrible. A big part of that is because nobody wants to touch Pascal/Delphi nowadays. I'm only comfortable with Pascal because I used in the 1998-2003. People are happier to test and contribute documentation, or work on the language bindings -- besides the projects here on DSS Extensions, I know people use it in Swift, Go, and a couple of people looking into Rust.

For example, using other programming languages, a lot of people develop features OpenDSS lacks and sometimes reimplement features they consider limited in the official engine; a lot of that goes to waste or is restricted to a small subset of users. There are other reasons for that, but I firmly believe that OpenDSS being written in Pascal is one of the main issues.

To workaround this Pascal issue, at the same time we've been working on finishing the C++ port to abandon Pascal completely (so, not a chance to even consider using the current official version of OpenDSS). Some of the changes in the engine incorporated here in DSS C-API will greatly facilitate that, and dss-extensions/dss_capi#88 is one of the final pieces -- like the rest of the big features, it will result in a major rewrite of the related code (and it's quite boring to work on), so I'm taking the opportunity to leave the code in a place that it's easier to port to C++. We already have a plan to address some concerns other people may have regarding this C++ port vs. the official version, so hopefully by next year we may have a more robust development community, and maybe (hopefully) more direct collaboration with EPRI.

[mackei]

Thank you for the extensive explanation!