example: fast, multi-threaded, non-blocking, port and host reuse, thread-safe, epoll server

[enghitalo]

Benchmark

v wipe-cache && v -prod -skip-unused crun .

Running 30s test @ http://localhost:3000/
  16 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.47ms    1.19ms  22.48ms   78.40%
    Req/Sec    23.05k     2.02k   53.61k    81.71%
  11017868 requests in 30.10s, 0.97GB read
Requests/sec: 366036.54
Transfer/sec:     33.16MB

with pontencial to be faster

[spytheman]

Please use something like examples/vanilla_http_server/, not examples/server/ .

Just server is too generic, and it does not make it clear, that it is an http server.

[spytheman]

Also add the new folder name in the efolders list in cmd/tools/vbuild-examples.v , so that v build-examples will know that it compiles as a project, and not each .v file in there, as a separate program (which is the reason for the current CI failures).

[enghitalo]

What we are missing to a 500k+ Requests/sec in

wrk -H 'Connection: keep-alive' --connections 512 --threads 8 --duration 5s --timeout 1 http://localhost:3000/

Unfortunately, I don't have much time to research and investigate bottlenecks. So, having the community's help would be amazing!!

[enghitalo]

wrk -H 'Connection: keep-alive' --connections 512 --threads 8 --duration 60s --timeout 1 http://localhost:3000/ 

Running 1m test @ http://localhost:3000/
  8 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.42ms    1.12ms  13.48ms   78.10%
    Req/Sec    47.73k     1.74k   51.86k    75.52%
  22792466 requests in 1.00m, 2.02GB read
Requests/sec: 379530.05
Transfer/sec:     34.39MB

[enghitalo]

NOTE: Removing all lockers and runnig with -gc none We have 425k+ Requests/sec. But running valgrind --tool=helgrind ./vanilla_http_server we will get a lot off errors what means that Garbage Collector is not working well with multi-threading. Then some freak workarounds like

(*server).lock_flag.lock()
mut readed_request_buffer := []u8{cap: bytes_read}
(*server).lock_flag.unlock()
defer {
	unsafe {
		(*server).lock_flag.lock()
		readed_request_buffer.free()
		(*server).lock_flag.unlock()
	}
}
unsafe {
	readed_request_buffer.push_many(&request_buffer[0], bytes_read)
}

Needs to be made :|

[enghitalo]

I'm very busy this week. I'll try to complete this task as soon as possible.

[JalonSolov]

It makes a lot more sense to use manual memory management for something low-level like this, where you want the best possible throughput.

[spytheman]

I am more concerned about it not compiling on windows.

Examples should either compile everywhere (ideally), or be skipped on the platforms, where they are known to fail in cmd/tools/vbuild-examples.v, otherwise the CI will be broken.

[enghitalo]

I am more concerned about it not compiling on windows.

Examples should either compile everywhere (ideally), or be skipped on the platforms, where they are known to fail in cmd/tools/vbuild-examples.v, otherwise the CI will be broken.

I agree with you. It will be compiled in all OSs

[enghitalo]

I will try to make it ready to merge today :D

[enghitalo]

v -prod crun examples/vanilla_http_server
wrk --connection 512 --threads 16 --duration 30s http://localhost:3000

Running 30s test @ http://localhost:3000
  16 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.43ms    1.10ms  27.64ms   75.69%
    Req/Sec    23.90k     1.57k   61.21k    84.72%
  11419029 requests in 30.10s, 1.00GB read
Requests/sec: 379380.28
Transfer/sec:     34.01MB

[JalonSolov]

Very nice speed increase.

[enghitalo]

445k
v wipe-cache && v -prod crun examples/vanilla_http_server
wrk --connection 512 --threads 16 --duration 30s http://localhost:3000

Running 30s test @ http://localhost:3000
  16 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.20ms    0.90ms  10.85ms   73.81%
    Req/Sec    28.08k     1.57k   71.99k    84.06%
  13423554 requests in 30.10s, 1.18GB read
Requests/sec: 445963.68
Transfer/sec:     39.98MB

[medvednikov]

Amazing! Great job @enghitalo

[medvednikov]

@Casper64 check this out

[enghitalo]

Thank you, @medvednikov

[enghitalo]

We finally cross the 520k boundary. no race condition detected!!

Running 30s test @ http://localhost:3001
  16 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.10ms    1.12ms  26.43ms   85.85%
    Req/Sec    32.67k     3.29k   60.78k    70.80%
  15628885 requests in 30.10s, 1.37GB read
Requests/sec: 519212.27
Transfer/sec:     46.54MB

[enghitalo]

@spytheman , Sorry, I tried to implement some solutions to support Windows, but the results were terrible. It increased the size of the project a lot and I felt a bit uncomfortable simulating technologies and concepts from one platform on another. When I create a library, I will possibly use the native technologies of each O.S.
For the examples folder, I'll prefer to keep it as simple as possible so that it's more educational for users.

[medvednikov]

I agree. I don't think we need an epoll server on windows. Too many resources needed for 0.5% of servers that will run it.

No need to use windows servers to run V. Debugging can be done via wls.

[spytheman]

For the examples folder, I'll prefer to keep it as simple as possible so that it's more educational for users.

ok, I've added it to the skip list for windows.

[enghitalo]

ok, I've added it to the skip list for windows.

Thank you