Split .data folder #779

joaoandre-avaiga · 2024-01-30T01:28:24Z

No description provided.

trgiangdo · 2024-01-30T01:47:24Z

taipy/core/_entity/_migrate_cli.py

Do we need to add a new entry in the migrated Config?
There was only "storage_folder" before, do we need to add the "tp_storage_folder"

trgiangdo · 2024-01-30T09:27:39Z

taipy/core/config/core_section.py

@@ -28,9 +28,12 @@ class CoreSection(UniqueSection):

    Attributes:
        root_folder (str): Path of the base folder for the taipy application. The default value is "./taipy/"
-        storage_folder (str): Folder name used to store Taipy data. The default value is ".data/". It is used in
+        storage_folder (str): Folder name used to store user data. The default value is ".user_data/". It is used in


How about "data_folder"?

This folder is the entrypoint folder for a user to get and drop data. I prefer data_folder instead of storage_folder, to be honest, but then it will be a breaking change. Am I correct?
What do you think?

For the default value, I have two remarks:

Why not make the default value not start with a .. In the end, it is made to be accessed by the user. It is not a hidden folder.

Why prefix it by user? Why not just data?

This folder is the entrypoint folder for a user to get and drop data. I prefer data_folder instead of storage_folder, to be honest, but then it will be a breaking change. Am I correct?

Yes, I kept the config name the same to avoid a breaking change. My propose is to keep storage folder for the user data, so it doesn't have impact right now. We could change the name in 4.0, in the future.

Why not make the default value not start with a .. In the end, it is made to be accessed by the user. It is not a hidden folder.

I guess theres no reason to keep it hidden.

Why prefix it by user? Why not just data?

I think data may be a little to generic and could conflict with other things the user have. We do delete stuff from the data folder in some cases, so it may be more secure if it's not a common name.

trgiangdo · 2024-01-30T09:27:49Z

taipy/core/config/core_section.py

            conjunction with the *root_folder* attribute. That means the storage path is <root_folder><storage_folder>
-            (The default path is "./taipy/.data/").
+            (The default path is "./taipy/.user_data/").
+        tp_storage_folder (str): Folder name used to store Taipy data. The default value is ".taipy_data/". It is used


How about "entities_folder"?

Whatever the name of the config attribute, I would prefix it with taipy so it is explicit that it is for internal usage. Actually I would use taipy_folder directly, but I am ok with both taipy_storage_folder and taipy_entities_folder.
What do you think?

For the default value, I would go for .taipy instead of .taipy_data

trgiangdo · 2024-01-30T09:30:10Z

taipy/core/config/core_section.py

+            (The default path is "./taipy/.user_data/").
+        tp_storage_folder (str): Folder name used to store Taipy data. The default value is ".taipy_data/". It is used
+            in conjunction with the *root_folder* attribute. That means the storage path is
+            <root_folder><storage_folder> (The default path is "./taipy/.taipy_data/").


Are we sure about this? I don't think the .taipy folder ever exists

Im not sure. I kept the current text, just replaced .data with the new folder name. Maybe @jrobinAV can verify if this docstring is correct.

<storage_folder> should be renamed by <taipy_storage_folder> (or whatever the name of the attribute we chose).

Suggested change

<root_folder><storage_folder> (The default path is "./taipy/.taipy_data/").

<root_folder><taipy_storage_folder> (The default path is "./taipy/.taipy/").

florian-vuillemot · 2024-01-30T10:05:52Z

taipy/core/data/csv.py

@@ -122,12 +122,13 @@ def __init__(
            **properties,
        )
        self._path = properties.get(self.__PATH_KEY, properties.get(self.__DEFAULT_PATH_KEY))
+        if self._path and ".data" in self._path:


.data?
Can we at least put it as a const?

This is only to migrate old taipy projects to the new structure. If .data is on the datanode current path, we move the file to the new folder. I can put it in a constant.

jrobinAV

We need to converge on the config attribute names and their default values.

Otherwise, I believe it is good.

jrobinAV · 2024-02-05T12:34:42Z

taipy/core/config/core_section.py

            conjunction with the *root_folder* attribute. That means the storage path is <root_folder><storage_folder>
-            (The default path is "./taipy/.data/").
+            (The default path is "./taipy/.user_data/").
+        tp_storage_folder (str): Folder name used to store Taipy data. The default value is ".taipy_data/". It is used


Whatever the name of the config attribute, I would prefix it with taipy so it is explicit that it is for internal usage. Actually I would use taipy_folder directly, but I am ok with both taipy_storage_folder and taipy_entities_folder.
What do you think?

For the default value, I would go for .taipy instead of .taipy_data

jrobinAV · 2024-02-05T14:18:06Z

taipy/core/config/core_section.py

@@ -28,9 +28,12 @@ class CoreSection(UniqueSection):

    Attributes:
        root_folder (str): Path of the base folder for the taipy application. The default value is "./taipy/"
-        storage_folder (str): Folder name used to store Taipy data. The default value is ".data/". It is used in
+        storage_folder (str): Folder name used to store user data. The default value is ".user_data/". It is used in


This folder is the entrypoint folder for a user to get and drop data. I prefer data_folder instead of storage_folder, to be honest, but then it will be a breaking change. Am I correct?
What do you think?

For the default value, I have two remarks:

Why not make the default value not start with a .. In the end, it is made to be accessed by the user. It is not a hidden folder.

Why prefix it by user? Why not just data?

jrobinAV · 2024-02-05T14:27:10Z

taipy/core/config/core_section.py

+            (The default path is "./taipy/.user_data/").
+        tp_storage_folder (str): Folder name used to store Taipy data. The default value is ".taipy_data/". It is used
+            in conjunction with the *root_folder* attribute. That means the storage path is
+            <root_folder><storage_folder> (The default path is "./taipy/.taipy_data/").


<storage_folder> should be renamed by <taipy_storage_folder> (or whatever the name of the attribute we chose).

Suggested change

<root_folder><storage_folder> (The default path is "./taipy/.taipy_data/").

<root_folder><taipy_storage_folder> (The default path is "./taipy/.taipy/").

jrobinAV · 2024-02-05T14:30:16Z

taipy/core/config/core_section.py

+                <root_folder><storage_folder> (The default path is "./taipy/.user_data/").
+            tp_storage_folder (str): Folder name used to store Taipy data. The default value is ".taipy_data/". It is
+                used in conjunction with the *root_folder* attribute. That means the storage path is
+                <root_folder><storage_folder> (The default path is "./taipy/.taipy_data/").


You mean <tp_storage_folder>.

joaoandre-avaiga requested review from florian-vuillemot, trgiangdo, jrobinAV and toan-quach January 30, 2024 01:28

trgiangdo reviewed Jan 30, 2024

View reviewed changes

florian-vuillemot reviewed Jan 30, 2024

View reviewed changes

jrobinAV reviewed Feb 5, 2024

View reviewed changes

joaoandre-avaiga added 2 commits February 11, 2024 20:17

feat: add new config attribute for taipy data

5ef9a4e

feat: add new data folders to gitignore

f5a8b2a

joaoandre-avaiga force-pushed the feature/split-data-folder branch 2 times, most recently from 65ba756 to 195d847 Compare February 11, 2024 23:34

feat: add handler functions for new storage folder on core config

886ed47

joaoandre-avaiga force-pushed the feature/split-data-folder branch from 195d847 to 886ed47 Compare February 12, 2024 17:31

Merge branch 'develop' into feature/split-data-folder

d1a43d2

jrobinAV approved these changes Feb 13, 2024

View reviewed changes

joaoandre-avaiga merged commit f217f7b into develop Feb 14, 2024
56 of 62 checks passed

joaoandre-avaiga deleted the feature/split-data-folder branch February 14, 2024 01:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split .data folder #779

Split .data folder #779

joaoandre-avaiga commented Jan 30, 2024

trgiangdo Jan 30, 2024

trgiangdo Jan 30, 2024

jrobinAV Feb 5, 2024

joaoandre-avaiga Feb 5, 2024

trgiangdo Jan 30, 2024

jrobinAV Feb 5, 2024

trgiangdo Jan 30, 2024

joaoandre-avaiga Jan 31, 2024

jrobinAV Feb 5, 2024 •

edited

Loading

florian-vuillemot Jan 30, 2024

joaoandre-avaiga Jan 31, 2024

jrobinAV left a comment

jrobinAV Feb 5, 2024

jrobinAV Feb 5, 2024

jrobinAV Feb 5, 2024 •

edited

Loading

jrobinAV Feb 5, 2024 •

edited

Loading

	<root_folder><storage_folder> (The default path is "./taipy/.taipy_data/").
	<root_folder><taipy_storage_folder> (The default path is "./taipy/.taipy/").

Split .data folder #779

Split .data folder #779

Conversation

joaoandre-avaiga commented Jan 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrobinAV Feb 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrobinAV left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrobinAV Feb 5, 2024 • edited Loading

Choose a reason for hiding this comment

jrobinAV Feb 5, 2024 • edited Loading

Choose a reason for hiding this comment

jrobinAV Feb 5, 2024 •

edited

Loading

jrobinAV Feb 5, 2024 •

edited

Loading

jrobinAV Feb 5, 2024 •

edited

Loading