Test coverage

[leios]

Fae.jl is currently lacking tests. Here are some that we need:

• A test for each object (square, circle, sierpinski, barnsley)
  • These tests should be extended to the full functionality of the define_...(...) functions, so we should test define_square(...) and define_rectangle(...) as well as a few different configurations for each shape. Hutchinson objects off one pixel and missing lower right value #30 is a bit limiting here, so we should also tackle this.
• A test for fractal executable chaining. In other words, if we have 4 operators $H_1$ , $H_2$ , $H_3$ , and $H_4$ , we need to make sure that $(H_1, H_2)$ can create the correct object primitive and $(H_3, H_4)$ can create the correct postprocessing operation. Maybe start with a square and move it around with each operation, then check the pixels of the final image to make sure it's actually correct? If you do it on a 9x9 pixel grid, you could have $H_1$ create the square, $H_2$ move it to the right, $H_3$ move both squares up, and $H_4$ rotate both 90 degrees and move to the lower left. This would color every pixel except the top left and bottom right.
• Make sure all examples run (test currently exists, but fails for some reason on CI -- see below)
• GPU tests (Can now be run, but not on CI)
• Layering tests (just use a small array and check the output at certain elements to make sure it's fine)
  • FUMLayer, ColorLayer, FractalLayers as canvas
  • layering of all on top of each other
• Edge detection tests
  • postprocessing framework tests
• Tests for finding bounds and overlaps between regions
  • Note: not sure what to do about grids that are offset by non-integer amounts (for example: if one grid is shifted by 1.5 pixels)

Broken Tests:

• Certain tests cause segfaults on ROCArrays #31

[leios]

Important note: the chaos_test introduced in #24 causes a segfault on AMDGPU, but not on CUDA / CPU. This will take some digging to figure out, so I disabled it for AMD specifically.

[leios]

So I am not quite sure where to put this. In #28 I added a test to run all examples. It will run locally, but something about the julia namespace bugs out on the testsuite:

 Got exception outside of a @test
  MethodError: no method matching smear_example(::Int64, ::Int64, ::Int64; ArrayType=Array)
  Closest candidates are:
    smear_example(::Any, ::Any, ::Any; ArrayType, output_type) at ~/projects/Fae.jl/examples/smear.jl:3
  Stacktrace:
    [1] example_tests(ArrayType::Type{Array})
      @ Main ~/projects/Fae.jl/test/example_tests.jl:28
    [2] macro expansion
      @ ~/projects/Fae.jl/test/example_tests.jl:40 [inlined]
    [3] macro expansion
      @ ~/builds/julia-1.7.1/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
    [4] example_testsuite(ArrayType::Type{Array})
      @ Main ~/projects/Fae.jl/test/example_tests.jl:40
    [5] top-level scope
      @ REPL[59]:1
    [6] top-level scope
      @ ~/.julia/packages/CUDA/DfvRa/src/initialization.jl:52
    [7] eval
      @ ./boot.jl:373 [inlined]
    [8] eval_user_input(ast::Any, backend::REPL.REPLBackend)
      @ REPL ~/builds/julia-1.7.1/share/julia/stdlib/v1.7/REPL/src/REPL.jl:150
    [9] repl_backend_loop(backend::REPL.REPLBackend)
      @ REPL ~/builds/julia-1.7.1/share/julia/stdlib/v1.7/REPL/src/REPL.jl:244
   [10] start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
      @ REPL ~/builds/julia-1.7.1/share/julia/stdlib/v1.7/REPL/src/REPL.jl:229
   [11] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
      @ REPL ~/builds/julia-1.7.1/share/julia/stdlib/v1.7/REPL/src/REPL.jl:362
   [12] run_repl(repl::REPL.AbstractREPL, consumer::Any)
      @ REPL ~/builds/julia-1.7.1/share/julia/stdlib/v1.7/REPL/src/REPL.jl:349
   [13] (::Base.var"#930#932"{Bool, Bool, Bool})(REPL::Module)
      @ Base ./client.jl:394
   [14] #invokelatest#2
      @ ./essentials.jl:716 [inlined]
   [15] invokelatest
      @ ./essentials.jl:714 [inlined]
   [16] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
      @ Base ./client.jl:379
   [17] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:309
   [18] _start()
      @ Base ./client.jl:495
Test Summary:                  | Pass  Error  Total
running all examples on Arrays |    4      1      5

We get such errors at random, so I am not sure how to fix this right now.

Maybe take inspiration from: https://github.com/JuliaGPU/KernelAbstractions.jl/blob/master/test/examples.jl ?

I need to modify it to make sure we run the correct example with the ..._example(...) function.