Copy the instance variables into locals before accessing it in Spark

[polinankoleva]

Accessing fields of the outer object will reference the whole object:

class MyClass(object):
    def __init__(self):
        self.field = "Hello"
    def doStuff(self, rdd):
        return rdd.map(lambda s: self.field + s)

To avoid this issue, the simplest way is to copy field into a local variable instead of accessing it externally:

def doStuff(self, rdd):
    field = self.field
    return rdd.map(lambda s: field + s)