Skip to content

Commit

Permalink
Fix spark image to support testing
Browse files Browse the repository at this point in the history
Signed-off-by: Constantin M Adam <[email protected]>
  • Loading branch information
cmadam committed Oct 19, 2024
1 parent 7d97cef commit 87902ac
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion transforms/universal/fdedup/spark/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ RUN pip install --no-cache-dir -e .
COPY ./src/signature_calc_spark.py .

# copy some of the samples in
# COPY src/filter_local_spark.py local/
COPY src/signature_calc_transform_spark.py fdedup_transform_spark.py
COPY src/signature_calc_spark.py local/fdedup_local_spark.py

# copy test
COPY test/ test/
Expand All @@ -46,6 +47,7 @@ USER spark

# Set environment
ENV PYTHONPATH=${SPARK_HOME}/work-dir/:${SPARK_HOME}/work-dir/src/:${PYTHONPATH}
ENV PATH=${SPARK_HOME}/work-dir/.local/bin/:${PATH}

# Put these at the end since they seem to upset the docker cache.
ARG BUILD_DATE
Expand Down

0 comments on commit 87902ac

Please sign in to comment.