Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUILD][CORE] Extract the logging system into a subproject. #3119

Open
wants to merge 77 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
6ebf94f
Withdrawn changes from #2858 as they break ABI compat
Mar 5, 2024
3601861
[core] Restored logger perf improvements that were blocked up to 1.6.0
Mar 5, 2024
be1e774
[core] New implementation for the logging/formatting system
ethouris May 24, 2024
308af58
Fixed parts in non-default-enabled code parts
ethouris May 24, 2024
5ae2a03
Updated sfmt.h with reimplementation fixes
ethouris May 25, 2024
089dd51
Fixed heavy logging cases and atomics. Withdrawn changes for enums. U…
ethouris May 25, 2024
e37d37d
Provided a special version for atomic. Withdrawn unnecessary changes
ethouris May 26, 2024
6ce84fe
Changed more formatting usage to sfmt
ethouris May 26, 2024
8126c49
Removed ostringstream from utilities
ethouris May 26, 2024
216c1ed
Removed ostringstream use. Fixed C++03 problem with Ensure declaration
May 27, 2024
d8cebcd
Cleared out warn-errors for logging-off version
May 27, 2024
4e39fd7
Moved Printable out of C++11-dependent section (args... no longer nee…
May 27, 2024
5199746
Added extra version of snprintf for old Windows
ethouris May 27, 2024
4843143
Fixed the use of std::atomic
ethouris May 28, 2024
1a0eca4
Fixed the right atomic type used with logging. Fixed some reported sh…
ethouris May 28, 2024
9643702
Fixed a clang-reported warning (to trigger rebuilding)
ethouris May 28, 2024
f4ecbc1
Updated sfmt.h, fixed sync formatting
Jun 17, 2024
b90f540
Updated and merged
Jun 17, 2024
7660e92
Merge branch 'master' into dev-add-sfmt-for-logging
Jun 18, 2024
0a607c7
Some cosmetic fixes. Fixed the use of std::abs
Jun 18, 2024
d2ec1cf
Fixed usage of <cmath> with std
Jun 19, 2024
193fe39
Fixed correct includes for std::div
Jun 19, 2024
c81d4d6
Renamed sfmt.h and moved to srt namespace
Jun 20, 2024
0920fbf
[core] Setting up fmt with iomanip for logging system
Jun 25, 2024
f4088e4
Removed the use of sfmt obufstream
Jun 25, 2024
e48a6cb
Removed named reference to avoid dangling ref warning
Jun 25, 2024
2ba12a0
Added lacking include for sstream
Jun 25, 2024
f16fed7
Cleaned up previous usage of sfmt facilities
Jun 26, 2024
a49bdd9
Provided C++03 version of fmt
Jun 27, 2024
8b4d44c
Added some fixes as attempt to compile in C++03 (failed)
Jul 11, 2024
eed8c3f
Added separate C++03 compatible fmt API
Jul 18, 2024
4b6c56e
Removed commented-out previous code
Jul 18, 2024
e1617ea
Removed commented-out code
Jul 18, 2024
ac3d5d5
Moved overloaded operator << to inside the class
Jul 18, 2024
d1ae64d
Added C++11 flag that could be handled also on Windows
Jul 19, 2024
7472f05
Attempting to force C++11 in MICROSOFT to compile tools
Jul 19, 2024
4acdf00
Changed the way to require C++11 in apps.
Jul 19, 2024
d8bda01
Blocked unused function reported as warn/error
Jul 19, 2024
d0f472f
Merge branch 'master' into dev-add-custom-fmt-for-logging
Jul 22, 2024
061ec3b
Added UDL for fmt_stringview to simplify raw string format passing
Jul 22, 2024
1a6ce38
Added some more fixes with use of ofmt
Jul 23, 2024
05b5bb7
[maint] Removed VC 2013. Added VC 2022
Jul 25, 2024
a8664d2
Merge branch 'main-ci-remove-old-windows' into dev-add-custom-fmt-for…
Jul 26, 2024
880fa9d
Reworked the fmt API
Jul 26, 2024
7aa87a5
Merge branch 'master' into dev-restore-logger-changes
Sep 2, 2024
fc9482b
Really restored changes
Sep 2, 2024
fecce73
Updated and fixed
Sep 3, 2024
51350b1
Removed copyright notice blindly copied from another project
Sep 3, 2024
f0bfba8
Added more replacement for stringstream. Removed unused CIPAddress::show
Sep 11, 2024
8f82530
Updated and fixed
Sep 24, 2024
6106607
Merge branch 'master' into dev-add-custom-fmt-for-logging
ethouris Sep 24, 2024
3376247
Updated and fixed
Sep 27, 2024
a8dbce8
Merge branch 'master' into dev-add-custom-fmt-for-logging
Oct 18, 2024
59435c2
Merge branch 'master' into dev-add-custom-fmt-for-logging
Dec 6, 2024
8e74ad5
Initial rework of the logging system
Dec 9, 2024
571b6f3
Adjusted to the new configuration API
Dec 9, 2024
c91d9de
[DEV] Extracted logging system to a separate subproject
Feb 18, 2025
d025710
Merge branch 'master' into dev-extract-logging-system
Feb 18, 2025
f3eb220
Adjustment changes for C++98. Further changes in remaining files used…
Feb 20, 2025
892f0c3
Added missing generated files. Some naming fixes and cosmetic changes
Feb 24, 2025
16f7f6c
Attempting to fix some build errors
Feb 24, 2025
fe73ebb
Removed 'far' as identified (keyword in MSVC)
Feb 24, 2025
00d4ab5
Fixed external thread id provider for threadname
Feb 24, 2025
f6746bf
Fixed problems reported on MSVC
ethouris Feb 24, 2025
cea8217
Forcedully set default C++ std value for libs
Feb 24, 2025
8e58e94
Fixed the use of always defined ENABLE_LOGGING.
Feb 25, 2025
cbcc80e
Changed sync configuration to extra include file
Feb 25, 2025
e05369c
Fixed the use of hvu_sync.h in hvu_threadname.h
Feb 25, 2025
d1e968c
Applied some parallel fixes from other PR
Feb 25, 2025
facc4ad
Merge branch 'master' into dev-add-custom-fmt-for-logging
Feb 25, 2025
99ca9b4
Updated and fixed
Feb 26, 2025
05d82be
Updated from 2964
Feb 26, 2025
b956d8d
Added lacking file. Fixed wrong dependency on stdint
Feb 28, 2025
4f4111e
Merge branch 'dev-add-custom-fmt-for-logging' into dev-extract-loggin…
Feb 28, 2025
3d3f7d5
Updated and fixed.
Feb 28, 2025
c028797
Merge branch 'master' into dev-extract-logging-system
Mar 10, 2025
d793b63
Some cosmetic fixes
Mar 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Updated and fixed
Mikolaj Malecki committed Feb 26, 2025
commit 99ca9b467b2d4c017db329bfacc26691b1cc4e32
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -79,7 +79,7 @@ script:
export PKG_CONFIG_PATH=$(brew --prefix openssl)"/lib/pkgconfig";
cmake . -DCMAKE_BUILD_TYPE=$BUILD_TYPE $BUILD_OPTS -DENABLE_UNITTESTS="ON";
fi
- make -j$(nproc);
- make VERBOSE=1 -j$(nproc);
- if [ "$TRAVIS_COMPILER" != "x86_64-w64-mingw32-g++" ]; then
ulimit -c unlimited;
./test-srt -disable-ipv6;
6 changes: 4 additions & 2 deletions apps/srt-file-transmit.cpp
Original file line number Diff line number Diff line change
@@ -389,8 +389,10 @@ bool DoUpload(UriParser& ut, string path, string filename,
while (n > 0)
{
int st = tar->Write(buf.data() + shift, n, 0, out_stats);
Verb() << "Upload: " << n << " --> " << st
<< (!shift ? string() : "+" + Sprint(shift));
Verb("Upload: ", n, " --> ", st, VerbNoEOL);
if (shift)
Verb("+", shift, VerbNoEOL);
Verb();
if (st == SRT_ERROR)
{
cerr << "Upload: SRT error: " << srt_getlasterror_str()
3 changes: 1 addition & 2 deletions apps/transmitmedia.cpp
Original file line number Diff line number Diff line change
@@ -41,7 +41,6 @@
#include "srt_compat.h"
#include "verbose.hpp"


using namespace std;
using namespace srt;

@@ -203,7 +202,7 @@ void SrtCommon::InitParameters(string host, map<string,string> par)
if (transmit_chunk_size > SRT_LIVE_MAX_PLSIZE)
throw std::runtime_error("Chunk size in live mode exceeds 1456 bytes; this is not supported");

par["payloadsize"] = Sprint(transmit_chunk_size);
par["payloadsize"] = fmts(transmit_chunk_size);
}

// Assign the others here.
2 changes: 1 addition & 1 deletion apps/verbose.hpp
Original file line number Diff line number Diff line change
@@ -11,7 +11,7 @@
#ifndef INC_SRT_VERBOSE_HPP
#define INC_SRT_VERBOSE_HPP

#include <iostream>
#include "ofmt_iostream.h"
#include "sync.h"

namespace Verbose
2 changes: 1 addition & 1 deletion srtcore/api.cpp
Original file line number Diff line number Diff line change
@@ -1403,7 +1403,7 @@ int srt::CUDTUnited::groupConnect(CUDTGroup* pg, SRT_SOCKGROUPCONFIG* targets, i
for (size_t i = 0; i < g.m_config.size(); ++i)
{
HLOGC(aclog.Debug, log << "groupConnect: OPTION @" << sid << " #" << g.m_config[i].so);
error_reason = Sprint("group-derived option: #", g.m_config[i].so);
error_reason = fmtcat("group-derived option: #", g.m_config[i].so);
ns->core().setOpt(g.m_config[i].so, &g.m_config[i].value[0], (int)g.m_config[i].value.size());
}

6 changes: 3 additions & 3 deletions srtcore/core.cpp
Original file line number Diff line number Diff line change
@@ -6000,12 +6000,12 @@ bool srt::CUDT::frequentLogAllowed(size_t logid, const time_point& tnow, std::st
const int supr = m_aSuppressedMsg[logid];

if (supr > 0)
w_why = Sprint("++SUPPRESSED: ", supr);
w_why = fmtcat("++SUPPRESSED: ", supr);
m_aSuppressedMsg[logid] = 0;
}
else
{
w_why = Sprint("Too early - last one was ", FormatDuration<DUNIT_MS>(tnow - m_tsLogSlowDown[logid].load()));
w_why = fmtcat("Too early - last one was ", FormatDuration<DUNIT_MS>(tnow - m_tsLogSlowDown[logid].load()));
// Set YOUR OWN bit, atomically.
m_LogSlowDownExpired |= uint8_t(BIT(logid));
++m_aSuppressedMsg[logid];
@@ -9328,7 +9328,7 @@ void srt::CUDT::updateAfterSrtHandshake(int hsv)
{
ScopedLock glock (uglobal().m_GlobControlLock);
grpspec = m_parent->m_GroupOf
? Sprint(" group=$", m_parent->m_GroupOf->id())
? fmtcat(" group=$", m_parent->m_GroupOf->id())
: string();
}
#else
8 changes: 4 additions & 4 deletions srtcore/fec.cpp
Original file line number Diff line number Diff line change
@@ -533,7 +533,7 @@ void FECFilterBuiltin::ClipPacket(Group& g, const CPacket& pkt)

ClipData(g, length_net, kflg, timestamp_hw, pkt.data(), pkt.size());

fmtc x; x.hex();
IF_HEAVY_LOGGING(fmtc x = fmtc().hex());
HLOGC(pflog.Debug, log << "FEC DATA PKT CLIP: "
<< "FLAGS=" << fmt<unsigned>(kflg, x)
<< " LENGTH[ne]=" << fmt(length_net, x)
@@ -562,7 +562,7 @@ void FECFilterBuiltin::ClipControlPacket(Group& g, const CPacket& pkt)

ClipData(g, *length_clip, *flag_clip, timestamp_hw, payload, payload_clip_len);

fmtc x; x.hex();
IF_HEAVY_LOGGING(fmtc x = fmtc().hex());
HLOGC(pflog.Debug, log << "FEC/CTL CLIP: "
<< "FLAGS=" << fmt<unsigned>(*flag_clip, x)
<< " LENGTH[ne]=" << fmt(*length_clip, x)
@@ -586,7 +586,7 @@ void FECFilterBuiltin::ClipRebuiltPacket(Group& g, Receive::PrivPacket& pkt)

ClipData(g, length_net, kflg, timestamp_hw, pkt.buffer, pkt.length);

fmtc x; x.hex();
IF_HEAVY_LOGGING(fmtc x = fmtc().hex());
HLOGC(pflog.Debug, log << "FEC REBUILT DATA CLIP: "
<< "FLAGS=" << fmt<unsigned>(kflg, x)
<< " LENGTH[ne]=" << fmt(length_net, x)
@@ -769,7 +769,7 @@ void FECFilterBuiltin::PackControl(const Group& g, signed char index, SrtPacket&
pkt.hdr[SRT_PH_TIMESTAMP] = g.timestamp_clip;
pkt.hdr[SRT_PH_SEQNO] = seq;

fmtc x; x.hex();
IF_HEAVY_LOGGING(fmtc x = fmtc().hex());
HLOGC(pflog.Debug, log << "FEC: PackControl: hdr("
<< (total_size - g.payload_clip.size()) << "): INDEX="
<< int(index) << " LENGTH[ne]=" << fmt(g.length_clip, x)
90 changes: 83 additions & 7 deletions srtcore/ofmt.h
Original file line number Diff line number Diff line change
@@ -5,8 +5,32 @@
// for FILE type from stdio. It has nothing to do with the rest of the {fmt}
// library, except that it reuses the namespace.

#ifndef INC_SRT_IFMT_H
#define INC_SRT_IFMT_H
// USAGE:
//
// 1. Using iostream style:
//
// ofmtstream sout;
//
// sout << "Value: " << v << " (" << fmt(v, fmtc().hex().width(2).fillzero()) << ")\n";
//
// NOTE: When passing a string literal, consider using "Value"_V (C++11 only)
// or OFMT_RAWSTR("Value"). Unfortunately C++ doesn't distinguish "Value" and
// char [20] v = "Value"; both here contain "Value\0", but sizeof(v) for them
// returns the size of the allocated space, not size of the string. Although
// the compiler should expand strlen() in place for literals, note that it
// won't do it if optimizations are turned off.
//
// 2. Using variadic style:
//
// sout.print("Value: ", v, " (", fmt(v, fmtc().hex().width(2).fillzero()), ")\n");
//
//
// OFMT has also a potential to be used together with iostream, but it requires more
// definition support. This is only the basic fragment to be used with the logging system,
// hence it provides only a wrapper over std::stringstream.

#ifndef INC_SRT_OFMT_H
#define INC_SRT_OFMT_H

#include <string>
#include <cstring>
@@ -139,6 +163,15 @@ struct fmt_proxy
basic_fmtc<CharType> format_spec;

fmt_proxy(const Value& v, const basic_fmtc<CharType>& f): val(v), format_spec(f) {}

template <class OutStream>
void sendto(OutStream& os) const
{
std::stringstream tmp;
format_spec.apply(tmp);
tmp << val;
os << tmp.rdbuf();
}
};

template <typename Value>
@@ -148,6 +181,12 @@ struct fmt_simple_proxy
// Iostream manipulators should not be sent to the stream.
// use fmt() with fmtc() instead.
fmt_simple_proxy(const Value& v): val(v) {}

template <class OutStream>
void sendto(OutStream& os) const
{
os << val;
}
};

// !!! IMPORTANT !!!
@@ -199,6 +238,11 @@ inline internal::fmt_stringview fmt_rawstr(const char* dd, size_t ss)
return internal::fmt_stringview(dd, ss);
}

inline internal::fmt_stringview fmt_rawstr(const std::string& s)
{
return internal::fmt_stringview(s.data(), s.size());
}

template <class Value> inline
internal::fmt_simple_proxy<Value> fmt(const Value& val)
{
@@ -279,17 +323,14 @@ class ofmtstream
template<class ValueType>
ofmtstream& operator<<(const internal::fmt_simple_proxy<ValueType>& prox)
{
buffer << prox.val;
prox.sendto(buffer);
return *this;
}

template<class ValueType>
ofmtstream& operator<<(const internal::fmt_proxy<ValueType, char>& prox)
{
std::stringstream tmp;
prox.format_spec.apply(tmp);
tmp << prox.val;
buffer << tmp.rdbuf();
prox.sendto(buffer);
return *this;
}

@@ -359,6 +400,41 @@ std::string fmtcat(const Args&... args)
return out.str();
}

#else

// Provide fmtcat for C++03 for up to 4 parameters

// The 1-argument version is for logical consistency.
template <typename Arg1> inline
std::string fmtcat(const Arg1& arg1)
{
return fmts(arg1);
}

template <typename Arg1, typename Arg2> inline
std::string fmtcat(const Arg1& arg1, const Arg2& arg2)
{
ofmtstream out;
out << arg1 << arg2;
return out.str();
}

template <typename Arg1, typename Arg2, typename Arg3> inline
std::string fmtcat(const Arg1& arg1, const Arg2& arg2, const Arg3& arg3)
{
ofmtstream out;
out << arg1 << arg2 << arg3;
return out.str();
}

template <typename Arg1, typename Arg2, typename Arg3, typename Arg4> inline
std::string fmtcat(const Arg1& arg1, const Arg2& arg2, const Arg3& arg3, const Arg4& arg4)
{
ofmtstream out;
out << arg1 << arg2 << arg3 << arg4;
return out.str();
}

#endif

template <class Value> inline
4 changes: 2 additions & 2 deletions srtcore/queue.cpp
Original file line number Diff line number Diff line change
@@ -455,7 +455,7 @@ void srt::CSndQueue::init(CChannel* c, CTimer* t)

#if ENABLE_LOGGING
++m_counter;
const std::string thrname = Sprint("SRT:SndQ:w", m_counter.load());
const std::string thrname = fmtcat("SRT:SndQ:w", m_counter.load());
const char* thname = thrname.c_str();
#else
const char* thname = "SRT:SndQ";
@@ -1217,7 +1217,7 @@ void srt::CRcvQueue::init(int qsize, size_t payload, int version, int hsize, CCh

#if ENABLE_LOGGING
const int cnt = ++m_counter;
const std::string thrname = Sprint("SRT:RcvQ:w", cnt);
const std::string thrname = fmtcat("SRT:RcvQ:w", cnt);
#else
const std::string thrname = "SRT:RcvQ:w";
#endif
38 changes: 2 additions & 36 deletions srtcore/utilities.h
Original file line number Diff line number Diff line change
@@ -485,8 +485,8 @@ class FixedArray

void throw_invalid_index(int i) const
{
std::ostringstream ss;
ss << "Index " << i << "out of range";
srt::ofmtstream ss;
ss << OFMT_RAWSTR("Index ") << i << OFMT_RAWSTR("out of range");
throw std::runtime_error(ss.str());
}

@@ -574,23 +574,6 @@ inline auto Move(In& i) -> decltype(std::move(i)) { return std::move(i); }

// Gluing string of any type, wrapper for operator <<

template <class Stream>
inline Stream& Print(Stream& in) { return in;}

template <class Stream, class Arg1, class... Args>
inline Stream& Print(Stream& sout, Arg1&& arg1, Args&&... args)
{
sout << std::forward<Arg1>(arg1);
return Print(sout, args...);
}

template <class... Args>
inline std::string Sprint(Args&&... args)
{
srt::ofmtstream sout;
Print(sout, args...);
return sout.str();
}

// We need to use UniquePtr, in the form of C++03 it will be a #define.
// Naturally will be used std::move() so that it can later painlessly
@@ -669,23 +652,6 @@ class UniquePtr: public std::auto_ptr<T>
operator bool () const { return 0!= get(); }
};

// A primitive one-argument versions of Sprint and Printable
template <class Arg1>
inline std::string Sprint(const Arg1& arg)
{
return srt::fmts(arg);
}

// Ok, let's make another version with two arguments to sweeten
// a bit the API for C++03 users.
template <class Arg1, class Arg2>
inline std::string Sprint(const Arg1& arg, const Arg2& arg2)
{
srt::ofmtstream out;
out << arg << arg2;
return out.str();
}

template<typename Map, typename Key>
typename Map::mapped_type map_get(Map& m, const Key& key, typename Map::mapped_type def = typename Map::mapped_type())
{
10 changes: 5 additions & 5 deletions testing/testmedia.cpp
Original file line number Diff line number Diff line change
@@ -455,7 +455,7 @@ void SrtCommon::InitParameters(string host, string path, map<string,string> par)
if (transmit_chunk_size > SRT_LIVE_MAX_PLSIZE)
throw std::runtime_error("Chunk size in live mode exceeds 1456 bytes; this is not supported");

par["payloadsize"] = Sprint(transmit_chunk_size);
par["payloadsize"] = fmtcat(transmit_chunk_size);
}
}

@@ -476,10 +476,10 @@ void SrtCommon::InitParameters(string host, string path, map<string,string> par)
int version = srt::SrtParseVersion(v.c_str());
if (version == 0)
{
throw std::runtime_error(Sprint("Value for 'minversion' doesn't specify a valid version: ", v));
throw std::runtime_error(fmtcat("Value for 'minversion' doesn't specify a valid version: ", v));
}
par["minversion"] = Sprint(version);
Verb() << "\tFIXED: minversion = 0x" << std::hex << std::setfill('0') << std::setw(8) << version << std::dec;
par["minversion"] = fmtcat(version);
Verb("\tFIXED: minversion = 0x", fmt(version, fmtc().hex().fillzero().width(8)));
}
}

@@ -1052,7 +1052,7 @@ void SrtCommon::OpenGroupClient()
Verb("\t#", i, " [", c.token, "] ", c.host, ":", c.port, VerbNoEOL);
vector<string> extras;
if (c.weight)
extras.push_back(Sprint("weight=", c.weight));
extras.push_back(fmtcat("weight=", c.weight));

if (!c.source.empty())
extras.push_back("source=" + c.source.str());