Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upload-pack: No longer use hidden-refs as exclude_patterns #1866

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

SURA907
Copy link

@SURA907 SURA907 commented Feb 27, 2025

cc: Patrick Steinhardt [email protected]

Copy link

gitgitgadget bot commented Feb 27, 2025

There are issues in commit e188fe8:
upload-pack: No longer use hidden-refs as exclude_patterns
Commit checks stopped - the message is too short
Prefixed commit message must be in lower case
Commit not signed off

Copy link

gitgitgadget bot commented Feb 27, 2025

There are issues in commit 13ebc46:
upload-pack: No longer use hidden-refs as exclude_patterns
Commit checks stopped - the message is too short
Prefixed commit message must be in lower case
Commit not signed off

Copy link

gitgitgadget bot commented Feb 27, 2025

There are issues in commit 28f388f:
upload-pack: no longer use hidden-refs as exclude_patterns
Commit checks stopped - the message is too short
Commit not signed off

@SURA907
Copy link
Author

SURA907 commented Feb 27, 2025

/submit

Copy link

gitgitgadget bot commented Feb 27, 2025

Submitted as [email protected]

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-1866/SURA907/master-v1

To fetch this version to local tag pr-1866/SURA907/master-v1:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-1866/SURA907/master-v1

Copy link

gitgitgadget bot commented Feb 27, 2025

On the Git mailing list, Patrick Steinhardt wrote (reply to this):

On Thu, Feb 27, 2025 at 12:46:11PM +0000, SURA via GitGitGadget wrote:
> From: SURA <[email protected]>
> 
> Signed-off-by: SURA <[email protected]>
> ---
>     upload-pack: No longer use hidden-refs as exclude_patterns
> 
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1866%2FSURA907%2Fmaster-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1866/SURA907/master-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1866
> 
>  upload-pack.c | 15 +++------------
>  1 file changed, 3 insertions(+), 12 deletions(-)
> 
> diff --git a/upload-pack.c b/upload-pack.c
> index 728b2477fcc..9ae42a463a3 100644
> --- a/upload-pack.c
> +++ b/upload-pack.c
> @@ -609,21 +609,12 @@ static int allow_hidden_refs(enum allow_uor allow_uor)
>  static void for_each_namespaced_ref_1(each_ref_fn fn,
>  				      struct upload_pack_data *data)
>  {
> -	const char **excludes = NULL;
>  	/*
> -	 * If `data->allow_uor` allows fetching hidden refs, we need to
> -	 * mark all references (including hidden ones), to check in
> -	 * `is_our_ref()` below.
> -	 *
> -	 * Otherwise, we only care about whether each reference's object
> -	 * has the OUR_REF bit set or not, so do not need to visit
> -	 * hidden references.
> +	 * config transfer.hideRefs of upload-pack is diffient from arg exclude of for-each-ref,
> +	 * We should not set exclude_patterns here
>  	 */
> -	if (allow_hidden_refs(data->allow_uor))
> -		excludes = hidden_refs_to_excludes(&data->hidden_refs);
> -
>  	refs_for_each_namespaced_ref(get_main_ref_store(the_repository),
> -				     excludes, fn, data);
> +				     NULL, fn, data);
>  }

This message is missing any context _why_ we want to do this. For
background: setting up these exclude patterns for hidden references is
quite an important performance optimization in large repositories, so
disabling it just like that is not an option without a good reason to do
so.

So what is the issue that you see and why is this fix the solution for
that issue?

Patrick

Copy link

gitgitgadget bot commented Feb 27, 2025

User Patrick Steinhardt <[email protected]> has been added to the cc: list.

@SURA907
Copy link
Author

SURA907 commented Feb 27, 2025

On the Git mailing list, Patrick Steinhardt wrote (reply to this):

On Thu, Feb 27, 2025 at 12:46:11PM +0000, SURA via GitGitGadget wrote:
> From: SURA <[email protected]>
> 
> Signed-off-by: SURA <[email protected]>
> ---
>     upload-pack: No longer use hidden-refs as exclude_patterns
> 
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1866%2FSURA907%2Fmaster-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1866/SURA907/master-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1866
> 
>  upload-pack.c | 15 +++------------
>  1 file changed, 3 insertions(+), 12 deletions(-)
> 
> diff --git a/upload-pack.c b/upload-pack.c
> index 728b2477fcc..9ae42a463a3 100644
> --- a/upload-pack.c
> +++ b/upload-pack.c
> @@ -609,21 +609,12 @@ static int allow_hidden_refs(enum allow_uor allow_uor)
>  static void for_each_namespaced_ref_1(each_ref_fn fn,
>  				      struct upload_pack_data *data)
>  {
> -	const char **excludes = NULL;
>  	/*
> -	 * If `data->allow_uor` allows fetching hidden refs, we need to
> -	 * mark all references (including hidden ones), to check in
> -	 * `is_our_ref()` below.
> -	 *
> -	 * Otherwise, we only care about whether each reference's object
> -	 * has the OUR_REF bit set or not, so do not need to visit
> -	 * hidden references.
> +	 * config transfer.hideRefs of upload-pack is diffient from arg exclude of for-each-ref,
> +	 * We should not set exclude_patterns here
>  	 */
> -	if (allow_hidden_refs(data->allow_uor))
> -		excludes = hidden_refs_to_excludes(&data->hidden_refs);
> -
>  	refs_for_each_namespaced_ref(get_main_ref_store(the_repository),
> -				     excludes, fn, data);
> +				     NULL, fn, data);
>  }

This message is missing any context _why_ we want to do this. For
background: setting up these exclude patterns for hidden references is
quite an important performance optimization in large repositories, so
disabling it just like that is not an option without a good reason to do
so.

So what is the issue that you see and why is this fix the solution for
that issue?

Patrick

Oh, so sorry, I should merge email messages
See this: https://lore.kernel.org/git/CAD6AYr-ZC32VNfUfMB63H-rQRfTdV=VQfBm67i2mG+6GDCNxkQ@mail.gmail.com/T/#u

The following is the content of the email


Hello everyone

OS: Linux Mint 22
git version: v2.48.1

I found that packed refs are excluded by the transfer.hideRefs front
match, while loose refs use full match (when transfer.hideRefs ends
with '/', it is prefix match, which is normal)

When the server uses git, after setting transfer.hideRefs, the
references that the client can see before and after server repo gc are
different

It seems that 59c35fa accidentally damaged upload-pack when optimizing
git for-each-ref

It seems that there is no simple fix except rolling back this commit?

@SURA907
Copy link
Author

SURA907 commented Feb 27, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant