Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: update the difficulty levels #324

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AmjadHD
Copy link

@AmjadHD AmjadHD commented Oct 18, 2021

Difficulty levels are subjective, so opinions are welcomed.

@ee7 ee7 changed the title updated the difficulty levels config: update the difficulty levels Oct 18, 2021
@ee7
Copy link
Member

ee7 commented Oct 18, 2021

Hi. Thanks for working on this.

For background, the current numbers were made relative to the expected end state of the track. That is, I kept everything as "easy" because we implemented all the easiest exercises first - there are a lot of unimplemented exercises that will be harder than basically every implemented exercise, but easier than react, so I wanted a space to distinguish those "medium" exercises.

I haven't looked deeply at this PR's changes, but if we want difficulty to be relative to the current exercises, we'd can argue that any extra delineation is an improvement over the status quo. But as soon as we add more exercises, I think many of the new "medium" and "hard" will have to be downgraded. I don't mind that.


As I see it, the underlying questions are:

  1. Should difficulty levels be absolute, or relative within the track? For example: should the base difficulty level on the X86 Assembly track be higher than that on the Python track, if they both implemented the same exercises?
  2. and if "relative within the track", should they be relative to the currently implemented exercises, or relative to the unimplemented exercises?
  3. Is the difficulty supposed to be the difficulty for a student at the time when we expect them to undertake it, or the difficulty for an experienced user of the language?

I assume that the answers are:

  1. "relative within the track"
  2. "relative to the currently implemented exercises"
  3. "difficulty for the student if they undertake the exercises in the expected order"

This was probably discussed in the past, but as far as I know, the post-v3 answers aren't actually documented upstream.

@ErikSchierboom is this documented? If not, can we document it?


The difficulty values still do nothing but change whether they're labelled "easy", "medium" or "hard" on the website, right?

If so, I think I prefer for us to use only a subset of difficulties, at least until more exercises are implemented. If I recall correctly, some tracks do this. But I don't know if that's frowned upon nowadays.

If not, I'd suggest 1, 4, 7, and 10, like the Rust track seems to do. Then we just decide if an exercise should be labelled as "easy", "medium" or "hard", but give ourselves some space in the middle for "nearly easy" and "nearly hard". This should be easier to adjust when we add harder exercises.

Sorry for making this more complicated. Any thoughts on all of this @AmjadHD and @ynfle?

We can look at the exercise difficulty on other tracks for guidance, but they're not directly comparable.

That said, as an example nitpick, I don't think diamond should be labelled as "hard".

From some personal tooling (which I've always intended to eventually add to configlet info):

31 tracks implement the `diamond` Practice Exercise:
Difficulty : 3.6 ± 2.0
Min: 1, Max: 8
track difficulty
babashka 5
bash 4
c 3
cfml 1
clojure 5
csharp 8
dart 3
delphi 8
elixir 5
erlang 2
fsharp 8
go 4
haskell 2
java 4
javascript 5
kotlin 4
lua 4
nim 1
pharo-smalltalk 3
php 1
purescript 2
python 1
r 3
ruby 4
rust 4
scala 4
sml 1
swift 3
tcl 3
typescript 5
wren 1

@AmjadHD
Copy link
Author

AmjadHD commented Oct 18, 2021

I was thinking about exercism implementing a rating system: let the user, upon completion of the exercise, rate its difficulty. then calculate a mean or something, variables such as the track's language can be considered in the formula.

@ErikSchierboom
Copy link
Member

I was thinking about exercism implementing a rating system: let the user, upon completion of the exercise, rate its difficulty. then calculate a mean or something, variables such as the track's language can be considered in the formula.

We might be implementing something like this in the future, but no guarantees.

@ErikSchierboom
Copy link
Member

"relative within the track"
"relative to the currently implemented exercises"
"difficulty for the student if they undertake the exercises in the expected order"

That is correct.

Copy link
Member

@ErikSchierboom ErikSchierboom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Differentiating between the difficulties is nice, as we might allow filtering on them later and they'll also influence how much reputation is gained for completing an exercise.

@dlesnoff
Copy link

What prevent this PR from being merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants