Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Soft deleted bucket restore samples new4 #28243

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
5641446
WIP- able to fetch generation and list of soft deleted bucket
shubhangi-google Nov 25, 2024
926013c
wip- fetch soft deleted bucket and restore bucket
shubhangi-google Nov 27, 2024
8b1469b
refactor
shubhangi-google Nov 28, 2024
d528370
adding unit tests
shubhangi-google Dec 11, 2024
77cf517
refactor
shubhangi-google Dec 12, 2024
2e36027
update
shubhangi-google Dec 12, 2024
7603f63
fixing lint issues
shubhangi-google Dec 13, 2024
792bbc1
refactor
shubhangi-google Dec 13, 2024
830d745
removing unwanted changes
shubhangi-google Dec 16, 2024
89e75eb
removing commented code
shubhangi-google Dec 16, 2024
cde0f55
removing unwanted file change
shubhangi-google Dec 16, 2024
da9c77b
refactor
shubhangi-google Dec 16, 2024
e330331
refactor
shubhangi-google Dec 18, 2024
903cdff
WIP-samples
shubhangi-google Dec 12, 2024
13d1508
adding samples and tests
shubhangi-google Dec 17, 2024
87eee24
refactor
shubhangi-google Dec 18, 2024
5963369
refactor
shubhangi-google Dec 19, 2024
b6b1df8
updating comments
shubhangi-google Dec 19, 2024
037bebf
fix lint issues
shubhangi-google Dec 20, 2024
ff16cee
fix lint issues
shubhangi-google Dec 20, 2024
e8d29bf
fix review comments
shubhangi-google Dec 21, 2024
c443ec7
removing pry
shubhangi-google Dec 21, 2024
179de7a
fix lint
shubhangi-google Dec 21, 2024
5ff1a78
try cli issue fix
shubhangi-google Dec 23, 2024
6e8a746
fix lint
shubhangi-google Dec 23, 2024
2219935
removing unwanted code
shubhangi-google Jan 15, 2025
b74f4c2
adding check to see if bucket is deleted or not
shubhangi-google Jan 15, 2025
be93c86
debugging
shubhangi-google Jan 15, 2025
8f54ec8
debugging
shubhangi-google Jan 15, 2025
7a18356
debugging
shubhangi-google Jan 15, 2025
cf5ed47
debugging
shubhangi-google Jan 15, 2025
ff431b8
debugging
shubhangi-google Jan 15, 2025
c2744b5
debugging
shubhangi-google Jan 15, 2025
d591365
debugging
shubhangi-google Jan 15, 2025
3b9b9d5
updating gemspec
shubhangi-google Jan 16, 2025
4c3f72e
debugging
shubhangi-google Jan 17, 2025
82c2bb6
debugging
shubhangi-google Jan 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion google-cloud-storage/google-cloud-storage.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Gem::Specification.new do |gem|
gem.add_dependency "google-cloud-core", "~> 1.6"
gem.add_dependency "google-apis-core", "~> 0.13"
gem.add_dependency "google-apis-iamcredentials_v1", "~> 0.18"
gem.add_dependency "google-apis-storage_v1", "~> 0.38"
gem.add_dependency "google-apis-storage_v1", "~> 0.40"
gem.add_dependency "googleauth", "~> 1.9"
gem.add_dependency "digest-crc", "~> 0.4"
gem.add_dependency "addressable", "~> 2.8"
Expand Down
25 changes: 25 additions & 0 deletions google-cloud-storage/lib/google/cloud/storage/bucket.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2280,6 +2280,31 @@ def signed_url path = nil,
end
end

# Fetches generation no. of bucket
# @example
# require "google/cloud/storage"
# storage = Google::Cloud::Storage.new
# bucket = storage.bucket "my-bucket"
# generation= bucket.generation
def generation
@gapi.generation
end

# Fetches soft_delete_time of a soft deleted bucket
# @example
# bucket.delete
# bucket.soft_delete_time
def soft_delete_time
@gapi.soft_delete_time
end

# Fetches hard_delete_time of a soft deleted bucket
# @example
# bucket.hard_delete_time
def hard_delete_time
@gapi.hard_delete_time
end

##
# Generate a PostObject that includes the fields and URL to
# upload objects via HTML forms.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ def all request_limit: nil, &block
# @private New Bucket::List from a Google API Client
# Google::Apis::StorageV1::Buckets object.
def self.from_gapi gapi_list, service, prefix = nil, max = nil,
user_project: nil
user_project: nil, soft_deleted: nil
buckets = new(Array(gapi_list.items).map do |gapi_object|
Bucket.from_gapi gapi_object, service, user_project: user_project
end)
Expand All @@ -155,6 +155,7 @@ def self.from_gapi gapi_list, service, prefix = nil, max = nil,
buckets.instance_variable_set :@prefix, prefix
buckets.instance_variable_set :@max, max
buckets.instance_variable_set :@user_project, user_project
buckets.instance_variable_set :@soft_deleted, soft_deleted
buckets
end

Expand Down
61 changes: 57 additions & 4 deletions google-cloud-storage/lib/google/cloud/storage/project.rb
Original file line number Diff line number Diff line change
Expand Up @@ -193,11 +193,20 @@ def add_custom_header header_name, header_value
# puts bucket.name
# end
#
def buckets prefix: nil, token: nil, max: nil, user_project: nil
# @example Retrieve soft deleted buckets
# require "google/cloud/storage"
#
# storage = Google::Cloud::Storage.new
#
# soft_deleted_buckets = storage.buckets soft_deleted: true
# soft_deleted_buckets.each do |bucket|
# puts bucket.name
# end
def buckets prefix: nil, token: nil, max: nil, user_project: nil, soft_deleted: nil
gapi = service.list_buckets \
prefix: prefix, token: token, max: max, user_project: user_project
prefix: prefix, token: token, max: max, user_project: user_project, soft_deleted: soft_deleted
Bucket::List.from_gapi \
gapi, service, prefix, max, user_project: user_project
gapi, service, prefix, max, user_project: user_project, soft_deleted: soft_deleted
end
alias find_buckets buckets

Expand All @@ -223,6 +232,10 @@ def buckets prefix: nil, token: nil, max: nil, user_project: nil
# account, transit costs will be billed to the given project. This
# parameter is required with requester pays-enabled buckets. The
# default is `nil`.
# @param [Integer] generation generation no of bucket
# on whether the bucket's current metageneration matches the given value.
# @param [Boolean] soft_deleted If true, returns the soft-deleted bucket.
# This parameter is required if generation is specified.
#
# The value provided will be applied to all operations on the returned
# bucket instance and its files.
Expand Down Expand Up @@ -256,9 +269,20 @@ def buckets prefix: nil, token: nil, max: nil, user_project: nil
# bucket = storage.bucket "other-project-bucket",
# user_project: "my-other-project"
# files = bucket.files # Billed to "my-other-project"
# @example With `soft_deleted` set to a true and generation specified:
# require "google/cloud/storage"
#
# storage = Google::Cloud::Storage.new
#
# bucket = storage.bucket "my-bucket",
# soft_deleted: true,
# generation: 1234567889
# puts bucket.name
#
def bucket bucket_name,
skip_lookup: false,
generation: nil,
soft_deleted: nil,
if_metageneration_match: nil,
if_metageneration_not_match: nil,
user_project: nil
Expand All @@ -269,7 +293,10 @@ def bucket bucket_name,
gapi = service.get_bucket bucket_name,
if_metageneration_match: if_metageneration_match,
if_metageneration_not_match: if_metageneration_not_match,
user_project: user_project
user_project: user_project,
soft_deleted: soft_deleted,
generation: generation

Bucket.from_gapi gapi, service, user_project: user_project
rescue Google::Cloud::NotFoundError
nil
Expand Down Expand Up @@ -554,6 +581,32 @@ def hmac_keys service_account_email: nil, project_id: nil,
max: max, user_project: user_project
end

##
# Restores a soft deleted bucket with bucket name and generation.
#
# @param [String] bucket_name Name of the bucket.
# @param [Fixnum] generation generation of the bucket.
#
# @return [Google::Cloud::Storage::Bucket, nil] Returns nil if bucket
# does not exist
#
# @example
# require "google/cloud/storage"
#
# storage = Google::Cloud::Storage.new
# generation= 123
#
# bucket = storage.restore_bucket "my-bucket", generation
# puts bucket.name
#
def restore_bucket bucket_name,
generation,
options: {}
gapi = service.restore_bucket bucket_name, generation,
options: options
Bucket.from_gapi gapi, service
end

##
# Generates a signed URL. See [Signed
# URLs](https://cloud.google.com/storage/docs/access-control/signed-urls)
Expand Down
20 changes: 18 additions & 2 deletions google-cloud-storage/lib/google/cloud/storage/service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,12 @@ def project_service_account

##
# Retrieves a list of buckets for the given project.
def list_buckets prefix: nil, token: nil, max: nil, user_project: nil, options: {}
def list_buckets prefix: nil, token: nil, max: nil, user_project: nil, soft_deleted: nil, options: {}
execute do
service.list_buckets \
@project, prefix: prefix, page_token: token, max_results: max,
user_project: user_project(user_project), options: options
user_project: user_project(user_project),
soft_deleted: soft_deleted, options: options
end
end

Expand All @@ -112,12 +113,16 @@ def get_bucket bucket_name,
if_metageneration_match: nil,
if_metageneration_not_match: nil,
user_project: nil,
soft_deleted: nil,
generation: nil,
options: {}
execute do
service.get_bucket bucket_name,
if_metageneration_match: if_metageneration_match,
if_metageneration_not_match: if_metageneration_not_match,
user_project: user_project(user_project),
soft_deleted: soft_deleted,
generation: generation,
options: options
end
end
Expand Down Expand Up @@ -654,6 +659,17 @@ def delete_file bucket_name,
end
end

##
# Restore soft deleted bucket
def restore_bucket bucket_name,
generation,
options: {}
execute do
service.restore_bucket bucket_name, generation,
options: options
end
end

##
# Restores a soft-deleted object.
def restore_file bucket_name,
Expand Down
33 changes: 33 additions & 0 deletions google-cloud-storage/samples/acceptance/buckets_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,10 @@
require_relative "../storage_get_public_access_prevention"
require_relative "../storage_get_requester_pays_status"
require_relative "../storage_get_retention_policy"
require_relative "../storage_get_soft_deleted_bucket"
require_relative "../storage_get_uniform_bucket_level_access"
require_relative "../storage_list_buckets"
require_relative "../storage_list_soft_deleted_buckets"
require_relative "../storage_lock_retention_policy"
require_relative "../storage_remove_bucket_label"
require_relative "../storage_remove_cors_configuration"
Expand Down Expand Up @@ -119,6 +121,37 @@
end
end

describe "storage_soft_deleted_bucket" do
let(:new_bucket_name) { random_bucket_name }

it "get soft deleted bucket, its soft_delete_time and hard_delete_time" do
new_bucket = storage_client.create_bucket new_bucket_name
new_generation = new_bucket.generation
grant_storage_permission new_bucket_name
# Check if the bucket exist
assert(new_bucket.exists?, "Bucket #{new_bucket_name} should exist")
delete_bucket_helper new_bucket_name
# Check if the bucket does not exist
deleted_bucket =storage_client.bucket new_bucket_name
refute(deleted_bucket, "Bucket #{new_bucket_name} should not exist")
deleted_bucket= storage_client.bucket new_bucket_name, generation: new_generation, soft_deleted: true

# _out, _err = capture_io do
# get_soft_deleted_bucket bucket_name: new_bucket_name, generation: new_generation
# end
assert deleted_bucket.soft_delete_time , "Bucket soft_delete_time should be present"
assert deleted_bucket.hard_delete_time , "Bucket hard_delete_time should be present"
end

it "lists soft deleted buckets" do
list_deleted_bucket, _err = capture_io do
list_soft_deleted_buckets
end
assert list_deleted_bucket, "List of soft deleted bucket should not be blank"
end
end


describe "storage_create_bucket_dual_region" do
it "creates dual region bucket" do
location = "US"
Expand Down
17 changes: 17 additions & 0 deletions google-cloud-storage/samples/acceptance/helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,23 @@ def create_bucket_helper bucket_name
end
end

def grant_storage_permission bucket_name

storage_client = Google::Cloud::Storage.new
bucket= storage_client.bucket bucket_name

object_viewer = "roles/storage.objectViewer"
member = "serviceAccount:542339357638-cr0dserr2evg7sv1meghqeu703274f3h@developer.gserviceaccount.com"
bucket.policy requested_policy_version: 3 do |policy|
policy.version = 3
policy.bindings.insert(
role: object_viewer,
members: member
)
end

end

def delete_bucket_helper bucket_name
storage_client = Google::Cloud::Storage.new
retry_resource_exhaustion do
Expand Down
15 changes: 15 additions & 0 deletions google-cloud-storage/samples/acceptance/project_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

require_relative "helper"
require_relative "../storage_get_service_account"
require_relative "../storage_restore_bucket"

describe "Storage Quickstart" do
let(:project) { Google::Cloud::Storage.new }
Expand All @@ -28,3 +29,17 @@
assert_includes out, "@gs-project-accounts.iam.gserviceaccount.com"
end
end

describe "storage_soft_deleted_bucket" do
let(:storage_client) { Google::Cloud::Storage.new }
let(:bucket) { fixture_bucket }
let(:generation) { bucket.gapi.generation }

it "restores a soft deleted bucket" do
delete_bucket_helper bucket.name
_out, _err = capture_io do
restore_bucket bucket_name: bucket.name, generation: generation
end
assert "soft_delete_time", "#{bucket.name} Bucket restored"
end
end
41 changes: 41 additions & 0 deletions google-cloud-storage/samples/storage_get_soft_deleted_bucket.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START storage_get_soft_deleted_bucket]
def get_soft_deleted_bucket bucket_name:, generation:
# The ID of your GCS bucket
# bucket_name = "your-unique-bucket-name"
# The generation no of your GCS bucket
# generation = "1234567896987"

require "google/cloud/storage"

storage = Google::Cloud::Storage.new
# fetching soft deleted bucket with soft_delete_time and hard_delete_time
deleted_bucket_fetch = storage.bucket bucket_name, generation: generation, soft_deleted: true

soft_delete_time = deleted_bucket_fetch.soft_delete_time
hard_delete_time = deleted_bucket_fetch.hard_delete_time

if (soft_delete_time && hard_delete_time).nil?
puts "Not Found"
else
puts "soft_delete_time - #{soft_delete_time}"
puts "hard_delete_time - #{hard_delete_time}"
end
end
# [END storage_get_soft_deleted_bucket]


get_soft_deleted_bucket bucket_name: ARGV.shift, generation: ARGV.shift if $PROGRAM_NAME == __FILE__
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START storage_list_soft_deleted_buckets]
def list_soft_deleted_buckets
require "google/cloud/storage"

storage = Google::Cloud::Storage.new

# fetching soft deleted bucket list
deleted_buckets = storage.buckets soft_deleted: true

deleted_buckets.each do |bucket|
puts bucket.name
end
end
# [END storage_list_soft_deleted_buckets]

list_soft_deleted_buckets if $PROGRAM_NAME == __FILE__
Loading
Loading