CCPP code generator needs to parse all available suites if SUITES argument is not present

[climbfuji]

This currently applies to ccpp_prebuild.py only, but logic needs to be put in place for cap_gen.py in the future as well.

EMC has requested that the CCPP code generator compiles all available suites into the executable if the --suites=abc,xyz argument is not specified when calling ccpp_prebuild.py. In the case of ccpp_prebuild.py, this can easily be done, because the CCPP prebuild config contains the location of all suites (SUITES_DIR).

For cap_gen.py, we could implement logic to either list individual suites or a directory containing a number of suites, which would then get compiled into the executable.

Will be fixed by #292 for ccpp_prebuild.py.

[gold2718]

How should it do this search? We are already pulling suites from multiple repositories. Only the host model knows where all the suite files may be located, how can this search be hard coded into capgen?

[climbfuji]

How should it do this search? We are already pulling suites from multiple repositories. Only the host model knows where all the suite files may be located, how can this search be hard coded into capgen?

Not hardcoded. I think there should be two options: either pass a list of suites, or a list of directories containing suite definition files as arguments. Since suite definition files all have a fixed format suite_SUITENAME.xml, this can be done easily unless I am missing something.

[climbfuji]

Fixed in #292 for ccpp_prebuild.py.

[gold2718]

No, you are not missing something, just that the requested interface was not spelled out in the issue text. Currently, the --suites keyword has the following functionality:

Comma separated list of suite definition filenames to process
Filenames with a '.xml' suffix are treated as suite definition XML files
Other filenames are treated as containing a list of .xml filenames

We could extend this to be:

Comma separated list of suite definition filenames or directories to process
Filenames with a '.xml' suffix are treated as suite definition XML files
Other filenames are treated as containing a list of .xml filenames
A directory entry causes all files of the form, suite_<name>.xml,
in that directory to be processed

[climbfuji]

No, you are not missing something, just that the requested interface was not spelled out in the issue text. Currently, the --suites keyword has the following functionality:

Comma separated list of suite definition filenames to process
Filenames with a '.xml' suffix are treated as suite definition XML files
Other filenames are treated as containing a list of .xml filenames

We could extend this to be:

Comma separated list of suite definition filenames or directories to process
Filenames with a '.xml' suffix are treated as suite definition XML files
Other filenames are treated as containing a list of .xml filenames
A directory entry causes all files of the form, suite_<name>.xml,
in that directory to be processed

Sounds good. Note that ccpp_prebuild.py has

    if args.suites:
        sdfs = [ 'suite_{0}.xml'.format(x) for x in args.suites.split(',')]
    else:
        sdfs = None

The name of the suites (e.g. --suites=abc,xyz) get translated into the actual filenames sdfs = [ 'suite_abc.xml', 'suite_xyz.xml' ]. The argparse config however says

parser.add_argument('--suites',     action='store', help='suite definition files to use (comma-separated, without path)', default='')

which should be changed to say

parser.add_argument('--suites',     action='store', help='name of the suites to use', default='')

When we switch to cap_gen.py, we can make the transition to specifying a list of suite definition files or directories instead of having an optional argument --suites that takes the names of the suites (without suite_ and .xml).

[gold2718]

I have a question that relates to this feature (as well as scheme and host-model lists).
Currently, while it is an error (or should be) to have two schemes or suites with the same name, I am not making any checks on filenames.
While I log a warning if you specify the same file more than once, that check is based on the full pathname.
Should I also add a check for two files with the same name from two different paths?
This could happen if you specify two directories from which to gather suite files and one suite is in both directories.
My feeling is that I should log a warning and ignore the second file. This would be similar to filepath usage by GNU make where it takes the file from the first directory in its list and ignores any it finds later.

Thoughts? Do we require that a suite's filename match the suite name? Should I check suite names instead of suite filenames?