Logging long variables (e.g. lists) to the edf file

[Klemtonius]

Hi edwin,

I just found out that the messages that are logged to the edf file have a maximum length of 128 characters. As such, if you for instance want to log a list variable at the end of your experiment, you may well lose some information. According to SR research it's not something that can be fixed directly; (rather you'd have to log separate elements of the list in a for-loop, say).

Luckily I found this in time myself, but it may be important to implement something that warns people for this when they do log a long variable?

Cheers,
Josh

[esdalmaijer]

Thanks for the heads-up!

I'm not quite sure what the best behaviour would be. PyGaze could automatically split up the message into more than one string, but this would result in odd log entries. Alternatively, we could print a warning to the console, but this would likely be missed. Finally, we could throw an exception when someone tries to log more than 128 characters, but image this happening in the middle of an experiment: you wouldn't really care that the entire message wasn't logged, but would be royally ticked off by the unexpected crash.

What do you think?

[Klemtonius]

Hi Edwin,
I think the best option would be to split the message into more than one text line. Log entries would become a bit odd, but then again I think it's fine if people know what to expect. So there could be a clear warning and explanation in the help section for this function. Analyzing the eyelink logfile is a manual endeavour anyway (at least making scripts etc), so I don't think having to retrieve a variable from multiple lines would impose that much more of a hassle.
And yes, you would be ticked off if the experiment crashes halfway through; but on the other hand you would at least know, and make sure everything goes right after your pilot participant. Can you imagine how ticked off you would be if you never got that warning or crash, only to find out you lost important data after 30 participants? :p

CheersJosh
Date: Thu, 30 Apr 2015 03:07:32 -0700
From: [email protected]
To: [email protected]
CC: [email protected]
Subject: Re: [PyGaze] Logging long variables (e.g. lists) to the edf file (#35)

Thanks for the heads-up!

I'm not quite sure what the best behaviour would be. PyGaze could automatically split up the message into more than one string, but this would result in odd log entries. Alternatively, we could print a warning to the console, but this would likely be missed. Finally, we could throw an exception when someone tries to log more than 128 characters, but image this happening in the middle of an experiment: you wouldn't really care that the entire message wasn't logged, but would be royally ticked off by the unexpected crash.

What do you think?

—
Reply to this email directly or view it on GitHub.

[smathot]

By the way, if we're going to fix up the log() function anyway, it would be a nice occasion to fix two other annoying things:

1. Make sure that it's unicode safe, which means converting all str objects to unicode before passing them on to pylink. We would have to assume a default encoding for str objects (utf-8 probably), and mention this in the docs. This should both work:

exp.pygaze_eyetracker.log(u'été')
exp.pygaze_eyetracker.log('été') # Assuming utf-8

2. Quoting and escaping the log message would help during parsing. Right now you get:

MSG [TIMESTAMP] Some long string with "quotes"

Which could become:

MSG [TIMESTAMP] "Some long string with \"quotes\""

What do you think? It's not necessary, but it's a nice touch.