Mutable method receivers in closures

[einar-hjortdal]

In this program I have the following function

fn (mut c BaseClient) attempt_process(cmd Cmder, attempt int) ! {
	c.with_connection(fn [cmd] (mut cn pool.Connection) ! {
		cn.with_writer(fn [cmd] (mut wr proto.Writer) ! {
			write_cmd(mut wr, cmd)!
		})!
		cn.with_reader(cmd.read_reply)! // <--
	})!
}

The method call marked by the arrow, cn.with_reader(cmd.read_reply)! uses an immutable cmd struct passed as a function.
The compiler doesn't complain about the fact that cmd.read_reply mutates cmd.
The cmd.read_reply function is run and the cmd inside of it is changed as expected, but the changes do not show outside of its scope.
Is this the fault of the compiler or is this behavior normal?

[JalonSolov]

It's normal. Closures make copies of the variables.

If you want the value modified outside the closure, you have to pass a reference.

Of course, this still shouldn't work because the original is not marked mut.

One could argue that the copy should not be mutable either, since the original wasn't, and there's no mut on the copy.

[JalonSolov]

cmd inside the closure is a copy of cmd outside the closure. Since cmd is not mutable outside, the copy should not be mutable, either.

You also don't have to create a separate variable...

c.with_connection(fn [&cmd] (mut cn pool.Connection) ! {

[einar-hjortdal]

c.with_connection(fn [&cmd] (mut cn pool.Connection) ! {

fn (mut c BaseClient) attempt_process(mut cmd Cmder, attempt int) ! {
	c.with_connection(fn [&cmd] (mut cn pool.Connection) ! {
		cn.with_writer(fn [cmd] (mut wr proto.Writer) ! {
			write_cmd(mut wr, cmd)!
		})!
		cn.with_reader(cmd.read_reply)!
	})!
}

This actually results in an error

error: unexpected token `&`, expecting name

[JalonSolov]

Ok, sorry 'bout that. My other comments stand, though.

[Wajinn]

mut ref_cmd := &cmd // <--
c.with_connection(fn [cmd, ref_cmd] (mut cn pool.Connection) 

Why both cmd and ref_cmd? When you have created ref_cmd?

It's normal. Closures make copies of the variables.
If you want the value modified outside the closure, you have to pass a reference.

What JalonSolov stated here, is understandable. As to the way things presently work. Though he added how things might should work too, which may cause confusion.

Coachonko, do you want to modify outside the closure or not? It's unclear which is wanted, which adds to the confusion of those trying to clarify and follow along.

[einar-hjortdal]

Coachonko, do you want to modify outside the closure or not? It's unclear which is wanted, which adds to the confusion of those trying to clarify and follow along.

What I want to understand is this:

fn new_heap_struct() &SomeStruct {
	return &SomeStruct{}
}
fn main() {
	some_closure := fn [heap_struct](){
		heap_struct.val = 'set_by_closure'
	}
	some_closure()
}

This function does not compile. The compiler says that heap_struct must be set to mut.
Meanwhile in the code I posted above:

fn (mut c BaseClient) attempt_process(cmd Cmder) ! {
	c.with_connection(fn [cmd] (mut cn pool.Connection) ! {
		cn.with_writer(fn [cmd] (mut wr proto.Writer) ! {
			write_cmd(mut wr, cmd)!
		})!
		cn.with_reader(cmd.read_reply)! // <-- // cmd.val is not set
	})!
}

cmd.read_reply is passed as a function to the cn.with_reader function.
This code compiles without problems despite the fact that cmd.read_reply invokes this code:

fn (mut cmd StatusCmd) read_reply(mut rd proto.Reader) ! {
	cmd.val = rd.read_string()! // <-- cmd.val is set correctly
}

Why is there a difference?

P.S. I have attempted several changes and I cannot get cmd to change outside the closure. I tried mut and references, so maybe there is more to this.

[einar-hjortdal]

I have not found an answer this this question. I have rewritten the function from this form:

fn (mut c BaseClient) attempt_process(mut cmd Cmder) ! {
	c.with_connection(fn [mut cmd] (mut cn pool.Connection) ! {
		cn.with_writer(fn [cmd] (mut wr proto.Writer) ! {
			write_cmd(mut wr, cmd)!
		})!
		cn.with_reader(cmd.read_reply)!
	})!
}

To this form:

fn (mut c BaseClient) attempt_process(mut cmd Cmder) ! {
	c.with_connection(fn [mut cmd] (mut cn pool.Connection) ! {
		cn.with_writer(fn [cmd] (mut wr proto.Writer) ! {
			write_cmd(mut wr, cmd)!
		})!
		cn.with_reader(fn [mut cmd] (mut rd proto.Reader) ! {
			cmd.read_reply(mut rd)!
		})!
	})!
}

When the function is written this way, cmd (which is a &Cmder type) is correctly passed as a reference and its value changes outside of the scope of cn.with_reader.

I believe the whole problem is caused by passing a method as a function:
Passing the method as a function does not allow to mark mutable receivers. Somehow this allows the method to be called with a mutable copy of the receiver inside the scope of the method.

[einar-hjortdal]

Another unexpected behavior I have encountered is this:

fn (mut c BaseClient) attempt_process(mut cmd Cmder) ! {
	mut ref_cmd := &cmd 
	// error: `cmd` cannot be referenced outside `unsafe` blocks as it might be stored on stack. Consider wrapping the `.Cmder` object in a `struct` declared as `[heap]`.

The problem being that Cmder is an interface, StatusCmd is the struct being passed as Cmder here. StatusCmd is already defined as [heap] and the object is created by a function that returns &StatusCmd.
Is it even possible to wrap an interface into a struct? What is this error?

[einar-hjortdal]

This post is an attempt to provide more information and a way to reproduce what I have encountered. This attempt follows a conversations that I had with @spytheman
I have tried to isolate the problem, but I found it harder to do than expected. Therefore, I have created a branch with some relevant commits and in-code comments describing the problem.

Setup

git clone --branch bugrep_function-params-in-closures https://github.com/Coachonko/redis.git
cd redis
git checkout f99737dc1386c667932bbff9809df974692f3a86
v run src/cmd_test.v

Following these steps, everything should work.
The relevant function is found here

Problem 1: passing mutable method receiver with function parameter

git checkout 3a7d04e59a92416700a03191ad605f35daf6a65e
v run src/cmd_test.v

Following these steps, the console will show the output of 2 println.
The diff found here contains information about what has changed and why this is unexpected.

Problem 2: unexpected runtime memory access error when println

git checkout d6c7c3d30c1be492d764d583438fe7de39011918
v run src/cmd_test.v

Diff found here.
I do not know why this is happening and it confuses me, it may be completely unrelated to the problem number 1.

Problem 3: compiler recommending to wrap an interface into a heap struct

git checkout 773ce6a9ba310c7761f9370f83a3017ac6adc865
v run src/cmd_test.v

Diff found here.
As explained in the comments, I don't think this should happen. Please note line 122 too.

I don't know if any of these are bugs. I would like to understand why this code behaves the way that it does. Please let me know what is happening.