[FSTORE-1672] Allow multiple on-demand features to be returned from an on-demand transformation function and allow passing of local variables to a transformation function #439
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ddition of context variables
bubriks
approved these changes
Feb 3, 2025
| An on-demand transformation function may be created by associating a [transformation function](../transformation_functions.md) with a feature group. Each on-demand transformation function generates a single on-demand feature, which, by default, is assigned the same name as the associated transformation function. For instance, in the example below, the on-demand transformation function `transaction_age` produces an on-demand feature named transaction_age. Alternatively, the name of the resulting on-demand feature can be explicitly defined using the [`alias`](../transformation_functions.md#specifying-output-features–names-for-transformation-functions) function. | ||
|
|
||
| It is important to note that only one-to-one or many-to-one transformation functions are compatible with the creation of on-demand transformation functions. | ||
| An on-demand transformation function may be created by associating a [transformation function](../transformation_functions.md) with a feature group. Each on-demand transformation function can generate one or multiple on-demand features. If the on-demand transformation function returns a single feature, it is automatically assigned the same name as the transformation function. However, if it returns multiple features, they are by default named using the format `functionName_outputColumnNumber`. For instance, in the example below, the on-demand transformation function `transaction_age` produces an on-demand feature named `transaction_age` and the on-demand transformation function `stripped_strings` produces the on-demand features names `stripped_strings_0` and `stripped_strings_1`. Alternatively, the name of the resulting on-demand feature can be explicitly defined using the [`alias`](../transformation_functions.md#specifying-output-features–names-for-transformation-functions) function. |
There was a problem hiding this comment.
wouldn't it be functionName_outputTupleElementIndex instead of functionName_outputColumnNumber (maybe i am misunderstanding)
There was a problem hiding this comment.
Basically it is mean the same maybe there might be a better way of phrasing things, I gave it outputColumnNumber since a pandasDataFrame can also be return to return multiple features from a transformation function and in that case it depends on the column number.
In general each output feature is named based on the position in the output tuple or Dataframe.
manu-sj
added a commit
to manu-sj/logicalclocks.github.io
that referenced
this pull request
Feb 3, 2025
…n on-demand transformation function and allow passing of local variables to a transformation function (logicalclocks#439)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Documentation correponding to changes done in https://github.com/logicalclocks/hopsworks-ee/pull/2204 and logicalclocks/hopsworks-api#452