Casting integers to time.Duration is usually as incorrect as implicitly assigning integer expressions to constants, variables, fields, and parameters of type time.Duration. You can usually replace instances of time.Duration(x) with x * time.Nanoseconds if you really want to convert your type into nanoseconds.
There should be an analyzer flag which, when enabled, also forbids explicit casts to time.Duration.