feat(block I/O): Implement Block I/O tracing #197

cvonelm · 2021-12-14T14:31:49Z

Introduces the --block-io switch to record block I/O events

The implementation is discussed in detail in #196

This closes #196

cvonelm · 2021-12-17T12:19:28Z

TODO: Currently the size of the read/writes is in sectors.

Multiply by 512 byte (unform sector size Linux uses in the generic block layer) to get the real size of the read/writes.

bmario

I think the class now called perf::bio::Writer is a misnomer. It doesn't write events, it caches them. The class called Matcher is the actual writer here.

include/lo2s/config.hpp

include/lo2s/perf/bio/matcher.hpp

man/lo2s.1.pod

bmario · 2021-12-23T11:18:59Z

man/lo2s.1.pod

+
+=item B<--block-io>
+
+Record block I/O events. These are based on the block:block_rq_insert for begin events and block:block_rq_complete for end events specifically.


Suggested change

Record block I/O events. These are based on the block:block_rq_insert for begin events and block:block_rq_complete for end events specifically.

Record block I/O events using the block:block_rq_insert tracepoint for begin events and block:block_rq_complete tracepoint for end events.

bmario · 2021-12-23T11:19:38Z

src/config.cpp

@@ -120,6 +120,7 @@ void parse_program_options(int argc, const char** argv)
    auto& kernel_tracepoint_options = parser.group("Kernel tracepoint options");
    auto& x86_adapt_options = parser.group("x86_adapt options");
    auto& x86_energy_options = parser.group("x86_energy options");
+    auto& io_options = parser.group("I/O tracing options");


Suggested change

auto& io_options = parser.group("I/O tracing options");

auto& io_options = parser.group("I/O recording options");

bmario · 2021-12-23T11:30:20Z

src/monitor/main_monitor.cpp

+        matcher_ = std::make_unique<perf::bio::Matcher>(trace_);
+        for (const auto& cpu : Topology::instance().cpus())
+        {
+            bio_monitors_.emplace_back(std::make_unique<BioMonitor>(trace_, Cpu(cpu.id), matcher_));


Since C++17, emplace_back returns a reference, I'd rather use that to start()

bmario · 2021-12-23T11:32:21Z

src/perf/bio/writer.cpp

+    events_.push_back(BioEvent(makedev(record->dev >> 20, record->dev & ((1U << 20) - 1)),
+                               record->sector, sample->time, type_, mode, record->nr_sector));
+
+    if (events_.size() == 1000)


Shouldn't this compare to config.block_cache_size instead?

bmario · 2022-01-17T09:46:09Z

include/lo2s/monitor/bio_monitor.hpp

+
+    std::string group() const override
+    {
+        return "bio::BioMonitor";


Suggested change

return "bio::BioMonitor";

return "lo2s::BioMonitor";

Or does it add lo2s:: somewhere else?

In any case, I think the bio is redundant, isn't it?

bmario · 2022-01-17T09:47:46Z

include/lo2s/perf/bio/event_cacher.hpp

+#include <map>
+#include <vector>
+
+extern "C" namespace lo2s


Suggested change

extern "C" namespace lo2s

namespace lo2s

Why?

cvonelm · 2022-02-11T12:32:56Z

Use Nonblocking instead of blocking becaue it is more accurate to block I/O.
Measure inserts/issues and completes for a more complete view.
Rework exception handling to better support crashes in reader creation.

bmario · 2022-02-18T13:54:10Z

The generated trace isn't perfect yet. In particular, the system tree is a bit borked.

SYSTEM_TREE_NODE                       0  Name: "aldi" <5>, Class: "machine" <4>, Parent: UNDEFINED
[...]
SYSTEM_TREE_NODE                      50  Name: "/dev/sda" <3879>, Class: "block device" <3880>, Parent: "aldi" <0>
SYSTEM_TREE_NODE                      51  Name: "/dev/nvme1n1" <3881>, Class: "block device" <3880>, Parent: "aldi" <0>
[...]
LOCATION_GROUP                        33  Name: "block I/O events" <68>, Type: PROCESS, Parent: "aldi" <0>
LOCATION                               0  Name: "Block I/O events for /dev/sda" <74>, Type: CPU_THREAD, # Events: 27035, Group: "block I/O events" <33>
LOCATION                               1  Name: "Block I/O events for /dev/sda1" <75>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               2  Name: "Block I/O events for /dev/sda2" <76>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               3  Name: "Block I/O events for /dev/nvme1n1" <77>, Type: CPU_THREAD, # Events: 17380, Group: "block I/O events" <33>
LOCATION                               4  Name: "Block I/O events for /dev/nvme1n1p1" <78>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               5  Name: "Block I/O events for /dev/nvme0n1" <79>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               6  Name: "Block I/O events for /dev/nvme0n1p1" <80>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               7  Name: "Block I/O events for /dev/nvme0n1p2" <81>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               8  Name: "Block I/O events for /dev/nvme0n1p3" <82>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>
LOCATION                               9  Name: "Block I/O events for /dev/nvme0n1p4" <83>, Type: CPU_THREAD, # Events: 0, Group: "block I/O events" <33>

While there are system tree nodes for "/dev/sda" and "/dev/nvme1n1", there's none for "/dev/nvme0n1"
All locations referencing a single location group
The only location group directly references the root STN
The STNs for the block devices aren't used anywhere
"block I/O events" is sometimes in lower and sometimes in upper case

cvonelm · 2022-02-24T18:22:08Z

locations are now created on demand too. so there is not going to be a /dev/nvme1n1 location if there aren't any events for it
should every block device be its own location group?
depends on 2
They are used for the creation of IO_REGULAR_FILE's
Consistently uses lower case now

bmario · 2022-02-25T08:49:53Z

2. should every block device be its own location group?

Yes.

src/metric/plugin/metrics.cpp

include/lo2s/perf/bio/reader.hpp

include/lo2s/monitor/bio_monitor.hpp

cvonelm · 2022-02-25T12:34:54Z

Always write system tree node, locations, location_groups for disks, add partitions to location_group.

This information can be retrieved from /sys/dev/block/*/uevent, by using DISKSEQ and DEVTYPE

include/lo2s/util.hpp

src/util.cpp

bmario · 2022-03-09T14:23:23Z

~~Also, rebase on master~~

Nvm. I did it for you.

- introduce --block-io and --block-cache-size options - write insert, issue and complete as non blocking events - uses - write insert, issue and complete as a non blocking event - based on block:block_rq_insert block:block_rq_issue and block:block_rq_complete tracepoints

bmario requested changes Dec 23, 2021

View reviewed changes

cvonelm force-pushed the issue-194-part-1-bio-tracing branch from 06e97df to a271d55 Compare January 14, 2022 11:41

bmario reviewed Jan 17, 2022

View reviewed changes

bmario reviewed Feb 25, 2022

View reviewed changes

src/metric/plugin/metrics.cpp Outdated Show resolved Hide resolved

include/lo2s/perf/bio/reader.hpp Outdated Show resolved Hide resolved

include/lo2s/monitor/bio_monitor.hpp Outdated Show resolved Hide resolved

bmario requested changes Mar 7, 2022

View reviewed changes

bmario force-pushed the issue-194-part-1-bio-tracing branch from b696ce5 to 31f3464 Compare March 10, 2022 15:43

cvonelm force-pushed the issue-194-part-1-bio-tracing branch from 31f3464 to f07ef8b Compare March 11, 2022 11:50

Refactoring random bits and bobs

510868d

bmario merged commit ed1889f into master Mar 15, 2022

bmario deleted the issue-194-part-1-bio-tracing branch March 15, 2022 15:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(block I/O): Implement Block I/O tracing #197

feat(block I/O): Implement Block I/O tracing #197

cvonelm commented Dec 14, 2021

cvonelm commented Dec 17, 2021

bmario left a comment

bmario Dec 23, 2021

bmario Dec 23, 2021

bmario Dec 23, 2021

bmario Dec 23, 2021

bmario Jan 17, 2022

bmario Jan 17, 2022 •

edited

Loading

bmario Jan 17, 2022 •

edited

Loading

cvonelm commented Feb 11, 2022

bmario commented Feb 18, 2022 •

edited

Loading

cvonelm commented Feb 24, 2022

bmario commented Feb 25, 2022

cvonelm commented Feb 25, 2022

bmario commented Mar 9, 2022 •

edited

Loading


		=item B<--block-io>

		Record block I/O events. These are based on the block:block_rq_insert for begin events and block:block_rq_complete for end events specifically.

	Record block I/O events. These are based on the block:block_rq_insert for begin events and block:block_rq_complete for end events specifically.
	Record block I/O events using the block:block_rq_insert tracepoint for begin events and block:block_rq_complete tracepoint for end events.

	auto& io_options = parser.group("I/O tracing options");
	auto& io_options = parser.group("I/O recording options");

feat(block I/O): Implement Block I/O tracing #197

feat(block I/O): Implement Block I/O tracing #197

Conversation

cvonelm commented Dec 14, 2021

cvonelm commented Dec 17, 2021

bmario left a comment

Choose a reason for hiding this comment

bmario Dec 23, 2021

Choose a reason for hiding this comment

bmario Dec 23, 2021

Choose a reason for hiding this comment

bmario Dec 23, 2021

Choose a reason for hiding this comment

bmario Dec 23, 2021

Choose a reason for hiding this comment

bmario Jan 17, 2022

Choose a reason for hiding this comment

bmario Jan 17, 2022 • edited Loading

Choose a reason for hiding this comment

bmario Jan 17, 2022 • edited Loading

Choose a reason for hiding this comment

cvonelm commented Feb 11, 2022

bmario commented Feb 18, 2022 • edited Loading

cvonelm commented Feb 24, 2022

bmario commented Feb 25, 2022

cvonelm commented Feb 25, 2022

bmario commented Mar 9, 2022 • edited Loading

bmario Jan 17, 2022 •

edited

Loading

bmario Jan 17, 2022 •

edited

Loading

bmario commented Feb 18, 2022 •

edited

Loading

bmario commented Mar 9, 2022 •

edited

Loading