Skip to content

More fuzzing2 #1170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 62 commits into from
Jun 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
fbecc77
fix a fuzz issue with a trailing newline in long multiline strings in…
phlptp Jun 6, 2025
a7d1579
subcommand generation in the fuzzer
phlptp Jun 7, 2025
cf434a8
add subcommand modifiers to fuzzApp
phlptp Jun 8, 2025
9f2a6ee
add a slow fuzz file
phlptp Jun 8, 2025
0da0f35
fix issue with custom subsubcommands
phlptp Jun 8, 2025
2dbba03
add a slow fuzz file
phlptp Jun 9, 2025
fc78406
fix some infinite loop paths in add_subcommand
phlptp Jun 9, 2025
673a179
validation error
phlptp Jun 9, 2025
f8fbfa4
fix glitch with some specific conditions of positionals and trigger o…
phlptp Jun 9, 2025
8b9a9ae
validation error
phlptp Jun 10, 2025
91f7eb4
detect conflicting positional names
phlptp Jun 10, 2025
5f6cf9f
validation error
phlptp Jun 10, 2025
60fc0fb
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 10, 2025
a67c51b
include test in fuzzFails
phlptp Jun 10, 2025
5fe81e3
code cleanup
phlptp Jun 10, 2025
9c2605a
add more debugging output
phlptp Jun 10, 2025
e573d60
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 10, 2025
ddc0e8c
rework the detection of duplicate options
phlptp Jun 11, 2025
84384fa
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 11, 2025
aebcefb
fix some warnings
phlptp Jun 11, 2025
55c8ba9
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 11, 2025
814e90b
refactor the duplicate option code to detect options in layers of opt…
phlptp Jun 12, 2025
b3e82b6
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 12, 2025
f6a477c
clang-tidy fixes
phlptp Jun 12, 2025
c20c243
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 12, 2025
16358da
validation error
phlptp Jun 13, 2025
545758c
deal with edge case related to MultiOptionPolicy::Join for config files
phlptp Jun 13, 2025
0966592
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 13, 2025
8b516e5
better handle join option policy and config files
phlptp Jun 13, 2025
856ed5f
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 13, 2025
e32cb03
validation error
phlptp Jun 13, 2025
2c86d2b
add fuzz fail
phlptp Jun 13, 2025
cffac6f
fix issue with config and very specific field name
phlptp Jun 14, 2025
f98bd1c
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 14, 2025
f210e91
validation error
phlptp Jun 14, 2025
d71a496
deal with nans in the fuzzApp
phlptp Jun 14, 2025
8a465c0
add new terms on the dictionary
phlptp Jun 14, 2025
47ad767
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 14, 2025
44fd811
fix error
phlptp Jun 14, 2025
eea777c
fix some warnings on the fuzzer
phlptp Jun 14, 2025
fba69b2
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 14, 2025
a9d5ec3
invalid config error
phlptp Jun 14, 2025
fc44a6c
rework finding the options that match config parameters
phlptp Jun 14, 2025
5c70b0b
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 14, 2025
3d01d2b
validation check error
phlptp Jun 15, 2025
6f005b1
fix some cli11 issues on certain compilers
phlptp Jun 15, 2025
3d67323
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 15, 2025
b55daa1
handle summed arguments a bit better in config files
phlptp Jun 15, 2025
ac07284
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 15, 2025
073eb11
validation check error
phlptp Jun 15, 2025
0db1037
handle additional test and balance with the sum MultiOptionPolicy
phlptp Jun 15, 2025
fca67e9
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 15, 2025
c4c281d
validation check error
phlptp Jun 15, 2025
ffcb2ea
move some optionTypeTests to NumericTypeTest file
phlptp Jun 15, 2025
ea542f8
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 15, 2025
7d4f0c9
add check for line extension in multiline quotes and do it only for r…
phlptp Jun 15, 2025
a166bbf
validation check error
phlptp Jun 16, 2025
a597116
handle more edge cases for join MultiOptionPolicy
phlptp Jun 17, 2025
58a61a4
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 17, 2025
e875c21
update Config_inl.hpp
phlptp Jun 17, 2025
f300d66
style: pre-commit.ci fixes
pre-commit-ci[bot] Jun 17, 2025
a219963
more fixes
phlptp Jun 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
305 changes: 291 additions & 14 deletions fuzz/fuzzApp.cpp

Large diffs are not rendered by default.

20 changes: 19 additions & 1 deletion fuzz/fuzzApp.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#endif

#include <atomic>
#include <complex>
#include <iostream>
#include <memory>
#include <optional>
Expand Down Expand Up @@ -51,6 +52,18 @@ class stringWrapper {
std::string val{};
};

/// @brief class for extracting data for custom subcommand generation.
class SubcommandData {
public:
std::string name{""};
std::string description{""};
std::string modifiers{""};
std::string data{""};
std::size_t next{std::string::npos};
};

SubcommandData extract_subcomand_info(const std::string &description_string, std::size_t index);

class FuzzApp {
public:
FuzzApp() = default;
Expand All @@ -62,7 +75,9 @@ class FuzzApp {
std::size_t add_custom_options(CLI::App *app, const std::string &description_string);
/** modify an option based on string*/
static void modify_option(CLI::Option *opt, const std::string &modifier);

/** modify a subcommand based on characters in a string*/
static void modify_subcommand(CLI::App *app, const std::string &modifiers);
/** return true if the app itself support conversiont to config files*/
CLI11_NODISCARD bool supports_config_file() const { return !non_config_required; }
int32_t val32{0};
int16_t val16{0};
Expand Down Expand Up @@ -110,6 +125,9 @@ class FuzzApp {
doubleWrapper dwrap{0.0};
stringWrapper swrap{};
std::string buffer{};
std::complex<double> cv3{0.0, 0.0};
std::complex<float> cv4{0.0, 0.0};

int intbuffer{0};
std::atomic<double> doubleAtomic{0.0};

Expand Down
10 changes: 10 additions & 0 deletions fuzz/fuzz_dictionary1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
"--tup1"
"--tup2"
"--tup4"
"--cv3"
"--cv4"
"--dwrap"
"--iwrap"
"--vtup"
Expand Down Expand Up @@ -118,6 +120,8 @@
"tup1"
"tup2"
"tup4"
"cv3"
"cv4"
"dwrap"
"iwrap"
"swrap"
Expand Down Expand Up @@ -181,6 +185,12 @@
"<flag "
"<flag>"
"</flag>"
"<subcommand"
"<subcommand>"
"</subcommand>"
">-"
">--"
"modifiers="
"name="
"description="
"alias="
3 changes: 3 additions & 0 deletions include/CLI/Option.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ using callback_t = std::function<bool(const results_t &)>;

class Option;
class App;
class ConfigBase;

using Option_p = std::unique_ptr<Option>;
/// Enumeration of the multiOption Policy selection
Expand All @@ -51,6 +52,7 @@ enum class MultiOptionPolicy : char {
/// to share parts of the class; an OptionDefaults can copy to an Option.
template <typename CRTP> class OptionBase {
friend App;
friend ConfigBase;

protected:
/// The group membership
Expand Down Expand Up @@ -230,6 +232,7 @@ class OptionDefaults : public OptionBase<OptionDefaults> {

class Option : public OptionBase<Option> {
friend App;
friend ConfigBase;

protected:
/// @name Names
Expand Down
5 changes: 4 additions & 1 deletion include/CLI/StringTools.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,10 @@ CLI11_INLINE bool is_binary_escaped_string(const std::string &escaped_string);
CLI11_INLINE std::string extract_binary_string(const std::string &escaped_string);

/// process a quoted string, remove the quotes and if appropriate handle escaped characters
CLI11_INLINE bool process_quoted_string(std::string &str, char string_char = '\"', char literal_char = '\'');
CLI11_INLINE bool process_quoted_string(std::string &str,
char string_char = '\"',
char literal_char = '\'',
bool disable_secondary_array_processing = false);

/// This function formats the given text as a paragraph with fixed width and applies correct line wrapping
/// with a custom line prefix. The paragraph will get streamed to the given ostream.
Expand Down
182 changes: 98 additions & 84 deletions include/CLI/impl/App_inl.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -165,94 +165,103 @@ CLI11_INLINE Option *App::add_option(std::string option_name,
std::function<std::string()> func) {
Option myopt{option_name, option_description, option_callback, this, allow_non_standard_options_};

if(std::find_if(std::begin(options_), std::end(options_), [&myopt](const Option_p &v) { return *v == myopt; }) ==
std::end(options_)) {
if(myopt.lnames_.empty() && myopt.snames_.empty()) {
// if the option is positional only there is additional potential for ambiguities in config files and needs
// to be checked
std::string test_name = "--" + myopt.get_single_name();
if(test_name.size() == 3) {
test_name.erase(0, 1);
// do a quick search in current subcommand for options
auto res =
std::find_if(std::begin(options_), std::end(options_), [&myopt](const Option_p &v) { return *v == myopt; });
if(res != options_.end()) {
const auto &matchname = (*res)->matching_name(myopt);
throw(OptionAlreadyAdded("added option matched existing option name: " + matchname));
}
/** get a top level parent*/
const App *top_level_parent = this;
while(top_level_parent->name_.empty() && top_level_parent->parent_ != nullptr) {
top_level_parent = top_level_parent->parent_;
}

if(myopt.lnames_.empty() && myopt.snames_.empty()) {
// if the option is positional only there is additional potential for ambiguities in config files and needs
// to be checked
std::string test_name = "--" + myopt.get_single_name();
if(test_name.size() == 3) {
test_name.erase(0, 1);
}
// if we are in option group
const auto *op = top_level_parent->get_option_no_throw(test_name);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option positional name matches existing option: " + test_name));
}
// need to check if there is another positional with the same name that also doesn't have any long or
// short names
op = top_level_parent->get_option_no_throw(myopt.get_single_name());
if(op != nullptr && op->lnames_.empty() && op->snames_.empty()) {
throw(OptionAlreadyAdded("unable to disambiguate with existing option: " + test_name));
}
} else if(top_level_parent != this) {
for(auto &ln : myopt.lnames_) {
const auto *op = top_level_parent->get_option_no_throw(ln);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option matches existing positional option: " + ln));
}

auto *op = get_option_no_throw(test_name);
op = top_level_parent->get_option_no_throw("--" + ln);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option positional name matches existing option: " + test_name));
throw(OptionAlreadyAdded("added option matches existing option: --" + ln));
}
// need to check if there is another positional with the same name that also doesn't have any long or short
// names
op = get_option_no_throw(myopt.get_single_name());
if(op != nullptr && op->lnames_.empty() && op->snames_.empty()) {
throw(OptionAlreadyAdded("unable to disambiguate with existing option: " + test_name));
}
for(auto &sn : myopt.snames_) {
const auto *op = top_level_parent->get_option_no_throw(sn);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option matches existing positional option: " + sn));
}
} else if(parent_ != nullptr) {
for(auto &ln : myopt.lnames_) {
auto *op = parent_->get_option_no_throw(ln);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option matches existing positional option: " + ln));
}
op = top_level_parent->get_option_no_throw("-" + sn);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option matches existing option: -" + sn));
}
for(auto &sn : myopt.snames_) {
auto *op = parent_->get_option_no_throw(sn);
if(op != nullptr && op->get_configurable()) {
throw(OptionAlreadyAdded("added option matches existing positional option: " + sn));
}
}
if(allow_non_standard_options_ && !myopt.snames_.empty()) {

for(auto &sname : myopt.snames_) {
if(sname.length() > 1) {
std::string test_name;
test_name.push_back('-');
test_name.push_back(sname.front());
const auto *op = top_level_parent->get_option_no_throw(test_name);
if(op != nullptr) {
throw(OptionAlreadyAdded("added option interferes with existing short option: " + sname));
}
}
}
if(allow_non_standard_options_ && !myopt.snames_.empty()) {
for(auto &sname : myopt.snames_) {
if(sname.length() > 1) {
for(auto &opt : top_level_parent->get_options()) {
for(const auto &osn : opt->snames_) {
if(osn.size() > 1) {
std::string test_name;
test_name.push_back('-');
test_name.push_back(sname.front());
auto *op = get_option_no_throw(test_name);
if(op != nullptr) {
throw(OptionAlreadyAdded("added option interferes with existing short option: " + sname));
}
}
}
for(auto &opt : options_) {
for(const auto &osn : opt->snames_) {
if(osn.size() > 1) {
std::string test_name;
test_name.push_back(osn.front());
if(myopt.check_sname(test_name)) {
throw(OptionAlreadyAdded("added option interferes with existing non standard option: " +
osn));
}
test_name.push_back(osn.front());
if(myopt.check_sname(test_name)) {
throw(OptionAlreadyAdded("added option interferes with existing non standard option: " + osn));
}
}
}
}
options_.emplace_back();
Option_p &option = options_.back();
option.reset(new Option(option_name, option_description, option_callback, this, allow_non_standard_options_));
}
options_.emplace_back();
Option_p &option = options_.back();
option.reset(new Option(option_name, option_description, option_callback, this, allow_non_standard_options_));

// Set the default string capture function
option->default_function(func);
// Set the default string capture function
option->default_function(func);

// For compatibility with CLI11 1.7 and before, capture the default string here
if(defaulted)
option->capture_default_str();
// For compatibility with CLI11 1.7 and before, capture the default string here
if(defaulted)
option->capture_default_str();

// Transfer defaults to the new option
option_defaults_.copy_to(option.get());
// Transfer defaults to the new option
option_defaults_.copy_to(option.get());

// Don't bother to capture if we already did
if(!defaulted && option->get_always_capture_default())
option->capture_default_str();
// Don't bother to capture if we already did
if(!defaulted && option->get_always_capture_default())
option->capture_default_str();

return option.get();
}
// we know something matches now find what it is so we can produce more error information
for(auto &opt : options_) {
const auto &matchname = opt->matching_name(myopt);
if(!matchname.empty()) {
throw(OptionAlreadyAdded("added option matched existing option name: " + matchname));
}
}
// this line should not be reached the above loop should trigger the throw
throw(OptionAlreadyAdded("added option matched existing option name")); // LCOV_EXCL_LINE
return option.get();
}

CLI11_INLINE Option *App::set_help_flag(std::string flag_name, const std::string &help_description) {
Expand Down Expand Up @@ -841,8 +850,8 @@ CLI11_INLINE std::vector<Option *> App::get_options(const std::function<bool(Opt
std::end(options));
}
for(auto &subc : subcommands_) {
// also check down into nameless subcommands
if(subc->get_name().empty() && !subc->get_group().empty() && subc->get_group().front() == '+') {
// also check down into nameless subcommands and specific groups
if(subc->get_name().empty() || (!subc->get_group().empty() && subc->get_group().front() == '+')) {
auto subcopts = subc->get_options(filter);
options.insert(options.end(), subcopts.begin(), subcopts.end());
}
Expand Down Expand Up @@ -1539,7 +1548,8 @@ App::_add_flag_like_result(Option *op, const ConfigItem &item, const std::vector
return true;
}
if(static_cast<int>(inputs.size()) > op->get_items_expected_max() &&
op->get_multi_option_policy() != MultiOptionPolicy::TakeAll) {
op->get_multi_option_policy() != MultiOptionPolicy::TakeAll &&
op->get_multi_option_policy() != MultiOptionPolicy::Join) {
if(op->get_items_expected_max() > 1) {
throw ArgumentMismatch::AtMost(item.fullname(), op->get_items_expected_max(), inputs.size());
}
Expand Down Expand Up @@ -1608,11 +1618,6 @@ CLI11_INLINE bool App::_parse_single_config(const ConfigItem &item, std::size_t
}
if(op == nullptr) {
op = get_option_no_throw(item.name);
} else if(!op->get_configurable()) {
auto *testop = get_option_no_throw(item.name);
if(testop != nullptr && testop->get_configurable()) {
op = testop;
}
}
} else if(!op->get_configurable()) {
if(item.name.size() == 1) {
Expand All @@ -1621,14 +1626,17 @@ CLI11_INLINE bool App::_parse_single_config(const ConfigItem &item, std::size_t
op = testop;
}
}
if(!op->get_configurable()) {
auto *testop = get_option_no_throw(item.name);
if(testop != nullptr && testop->get_configurable()) {
op = testop;
}
}
if(op == nullptr || !op->get_configurable()) {
std::string iname = item.name;
auto options = get_options([iname](const CLI::Option *opt) {
return (opt->get_configurable() &&
(opt->check_name(iname) || opt->check_lname(iname) || opt->check_sname(iname)));
});
if(!options.empty()) {
op = options[0];
}
}

if(op == nullptr) {
// If the option was not present
if(get_allow_config_extras() == config_extras_mode::capture) {
Expand Down Expand Up @@ -1785,7 +1793,9 @@ CLI11_INLINE bool App::_parse_positional(std::vector<std::string> &args, bool ha
posOpt->add_result(std::string{});
}
}
results_t prev;
if(posOpt->get_trigger_on_parse() && posOpt->current_option_state_ == Option::option_state::callback_run) {
prev = posOpt->results();
posOpt->clear();
}
if(posOpt->get_expected_min() == 0) {
Expand All @@ -1801,6 +1811,10 @@ CLI11_INLINE bool App::_parse_positional(std::vector<std::string> &args, bool ha
if(posOpt->get_trigger_on_parse()) {
if(!posOpt->empty()) {
posOpt->run_callback();
} else {
if(!prev.empty()) {
posOpt->add_result(prev);
}
}
}

Expand Down
Loading
Loading