Skip to content

feat(bucket): add server-side copy and check rename and copy before the request #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ehennestad merged 3 commits into from
Nov 3, 2026
Merged

feat(bucket): add server-side copy and check rename and copy before the request #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
624155b
feat(bucket): add server-side copy and check rename and copy before t…
ehennestad Nov 2, 2026
f9e7d7d
fix(bucket): refuse a rename or copy onto itself, into itself, or acr…
ehennestad Nov 2, 2026
6d5f0a5
test(bucket): label the queued responses of the rename and copy tests
ehennestad Nov 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 86 additions & 9 deletions src/ebrains/+ebrains/+bucket/+api/BucketsClient.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
%
% Each method mirrors one endpoint under /buckets of the Data Proxy API:
% bucket stat, one page of the object listing, temporary download and
% upload URLs, the three requests of a multipart upload, rename, and
% delete.
% upload URLs, the three requests of a multipart upload, rename, copy,
% and delete.
% An object name that holds "/" (a folder within the bucket) is sent with
% the "/" as path separators, since the proxy takes the rest of the path
% as the name. The temporary URLs the proxy returns are made valid URLs
% before they are handed back; see the local functions in this file.
% as the name. The copy endpoint is the exception: copyObject sends the
% name as one segment with every "/" encoded, which its folder form
% needs. The temporary URLs the proxy returns are made valid URLs before
% they are handed back; see the local functions in this file.
%
% The functions of the ebrains.bucket namespace build on this client and
% are the intended entry point. Use the client directly when an endpoint
Expand All @@ -19,7 +21,7 @@
% bucketStat = client.getBucketStat("my-bucket");
%
% See also ebrains.bucket.listBucketObjects, ebrains.bucket.getBucketObject,
% ebrains.bucket.renameObject, ebrains.bucket.deleteObject
% ebrains.bucket.renameObject, ebrains.bucket.copyObject, ebrains.bucket.deleteObject

properties (Constant, Access = protected)
ErrorIdPrefix = "EBRAINS:Bucket"
Expand Down Expand Up @@ -258,18 +260,24 @@ function completeMultipartUpload(obj, bucketName, objectName, uploadId, partETag
end
end

function renameObject(obj, bucketName, objectName, targetName)
function renameObject(obj, bucketName, objectName, targetName, options)
% renameObject - Rename an object of a bucket
%
% client.renameObject(bucketName, objectName, targetName) gives
% the object the name targetName. To rename a folder, end
% objectName with "/".
% objectName with "/". The proxy queues the rename and completes
% it in the background; an object of the target name is replaced.
%
% client.renameObject(..., SendSuccessEmail=true) lets the proxy
% mail the account once the rename is done. The default is false.
% A rename that fails is reported by mail either way.

arguments
obj (1,1) ebrains.bucket.api.BucketsClient
bucketName (1,1) string {mustBeNonzeroLengthText}
objectName (1,1) string {mustBeNonzeroLengthText}
targetName (1,1) string {mustBeNonzeroLengthText}
options.SendSuccessEmail (1,1) logical = false
end

% The target travels in the body, where nothing changes it, but
Expand All @@ -282,15 +290,75 @@ function renameObject(obj, bucketName, objectName, targetName)
end

payload = struct('rename', struct('target_name', targetName));
requiredParams = struct('send_success_email', options.SendSuccessEmail);

apiUri = obj.buildApiUri(["buckets", bucketName, objectPathSegments(objectName)]);
apiUri = obj.buildApiUri(["buckets", bucketName, objectPathSegments(objectName)], requiredParams);
request = obj.initializeRequestMessage("PATCH", JSONPayload=jsonencode(payload));
response = obj.sendRequest(request, apiUri);

if response.StatusCode ~= "OK"
if ~isQueuedOrDone(response)
obj.throwError("renameObject", response)
end
end

function copyObject(obj, bucketName, objectName, targetName, options)
% copyObject - Copy an object of a bucket to another name in the same bucket
%
% client.copyObject(bucketName, objectName, targetName) writes a
% copy of the object under the name targetName and leaves the
% object itself in place. To copy a folder with everything in it,
% end objectName and targetName with "/". The proxy queues the
% copy and completes it in the background; an object of the
% target name is replaced.
%
% client.copyObject(..., SendSuccessEmail=true) lets the proxy
% mail the account once the copy is done. The default is false.
% A copy that fails is reported by mail either way.

arguments
obj (1,1) ebrains.bucket.api.BucketsClient
bucketName (1,1) string {mustBeNonzeroLengthText}
objectName (1,1) string {mustBeNonzeroLengthText}
targetName (1,1) string {mustBeNonzeroLengthText}
options.SendSuccessEmail (1,1) logical = false
end

% Neither name goes through the path check of buildApiUri: the
% source is sent as one segment (see below) and the target in
% the query. An object with such a name could not be reached
% afterwards, so both are refused here.
for name = [objectName, targetName]
if hasDotSegment(split(name, "/"))
error('EBRAINS:Bucket:DotSegmentInName', ...
['The name "%s" has a "." or ".." segment, so the copied object could not be ' ...
'reached by its name. Give names without "." or ".." between slashes.'], name)
end
end

requiredParams = struct('name', targetName, 'send_success_email', options.SendSuccessEmail);

% The endpoint is a PUT without a body, which MATLAB warns
% about on purpose. The warning is off for this request only.
warnState = warning('off', 'MATLAB:http:BodyExpectedFor');
warningCleanup = onCleanup(@() warning(warnState));

% The name is sent as a single segment, with every "/" encoded
% as %2F, unlike for the other endpoints (see objectPathSegments).
% A trailing "/" marks a folder, and sent as a path separator it
% would stand right before the "copy" segment, where the proxy
% collapses the two slashes into one before it reads the name:
% the copy then runs on an object named like the folder, which
% does not exist, and fails. The copy endpoint signs no URL
% with the name, so the encoded form reaches it intact.
apiUri = obj.buildApiUri(["buckets", bucketName, objectName, "copy"], requiredParams);
request = obj.initializeRequestMessage("PUT");
response = obj.sendRequest(request, apiUri);

if ~isQueuedOrDone(response)
obj.throwError("copyObject", response)
end
end

function deleteObject(obj, bucketName, objectName)
% deleteObject - Delete an object of a bucket
%
Expand Down Expand Up @@ -357,6 +425,15 @@ function deleteObject(obj, bucketName, objectName)
tf = any(segments == "." | segments == "..");
end

function tf = isQueuedOrDone(response)
% isQueuedOrDone - Whether the reply to a rename or copy means the proxy took the request
%
% The proxy answers 201 ("Your copy is processing") when it has queued
% the operation to complete in the background, and 200 when it is done
% within the request. Both mean the request was accepted.
tf = response.StatusCode == "OK" || response.StatusCode == "Created";
end

function segments = objectPathSegments(objectName)
% objectPathSegments - Path segments of an object name, one per folder level
%
Expand Down
61 changes: 61 additions & 0 deletions src/ebrains/+ebrains/+bucket/+internal/checkSourceAndTarget.m
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
function checkSourceAndTarget(client, bucketName, objectName, targetName, overwrite)
% checkSourceAndTarget - Refuse a rename or copy that the proxy would accept and then fail or misapply
%
% ebrains.bucket.internal.checkSourceAndTarget(client, bucketName,
% objectName, targetName, overwrite) throws
% EBRAINS:Bucket:SameName when the two names are equal
% EBRAINS:Bucket:FolderNameMismatch when one name ends with "/"
% (a folder) and the other does not
% EBRAINS:Bucket:TargetWithinSource when the target lies within the
% source folder, so the folder
% would go into itself
% EBRAINS:Bucket:ObjectNotFound when the bucket holds no object
% named objectName
% EBRAINS:Bucket:TargetExists when it holds one named
% targetName and overwrite is false
% A folder counts as present when any object is named below it.
%
% The Data Proxy queues a rename or copy and reports one that fails
% only by mail, so these checks are the only answer the caller gets at
% once. The name checks send nothing; the other two send listing
% requests, never the rename or copy.
%
% See also ebrains.bucket.internal.hasObject

arguments
client (1,1) ebrains.bucket.api.BucketsClient
bucketName (1,1) string {mustBeNonzeroLengthText}
objectName (1,1) string {mustBeNonzeroLengthText}
targetName (1,1) string {mustBeNonzeroLengthText}
overwrite (1,1) logical
end

if objectName == targetName
error('EBRAINS:Bucket:SameName', ...
'The target name "%s" is the name of the object itself. Give another target name.', targetName)
end

isFolder = endsWith(objectName, "/");
if isFolder ~= endsWith(targetName, "/")
error('EBRAINS:Bucket:FolderNameMismatch', ...
['The names "%s" and "%s" must both end with "/", for a folder, or neither, for an ' ...
'object.'], objectName, targetName)
end

if isFolder && startsWith(targetName, objectName)
error('EBRAINS:Bucket:TargetWithinSource', ...
'The target "%s" lies within the folder "%s", which would go into itself.', ...
targetName, objectName)
end

if ~ebrains.bucket.internal.hasObject(client, bucketName, objectName)
error('EBRAINS:Bucket:ObjectNotFound', ...
'Object "%s" was not found in bucket "%s".', objectName, bucketName)
end

if ~overwrite && ebrains.bucket.internal.hasObject(client, bucketName, targetName)
error('EBRAINS:Bucket:TargetExists', ...
['An object named "%s" is already in bucket "%s". Pass Overwrite=true to replace it, ' ...
'or give another target name.'], targetName, bucketName)
end
end
38 changes: 38 additions & 0 deletions src/ebrains/+ebrains/+bucket/+internal/hasObject.m
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
function tf = hasObject(client, bucketName, objectName)
% hasObject - Whether a bucket holds an object of a name, or any object below a folder name
%
% tf = ebrains.bucket.internal.hasObject(client, bucketName, objectName)
% lists the bucket with the name as prefix and returns true when an
% object has exactly that name. The prefix also matches longer names
% ("a.txt" and "a.txt.bak"), hence the exact-name comparison.
%
% A name that ends with "/" stands for a folder, and the function
% returns true when any object is named below it. The listing is
% narrowed to one object then, since one is enough to answer.
%
% See also ebrains.bucket.renameObject, ebrains.bucket.copyObject

arguments
client (1,1) ebrains.bucket.api.BucketsClient
bucketName (1,1) string {mustBeNonzeroLengthText}
objectName (1,1) string {mustBeNonzeroLengthText}
end

isFolderName = endsWith(objectName, "/");
if isFolderName
page = client.listObjects(bucketName, prefix=objectName, limit=1);
else
page = client.listObjects(bucketName, prefix=objectName);
end
objects = page.objects;

% jsondecode turns an empty JSON array into [], which has no name field,
% so an empty listing cannot go through the name comparison.
if isempty(objects)
tf = false;
elseif isFolderName
tf = true;
else
tf = any(strcmp({objects.name}, objectName));
end
end
63 changes: 63 additions & 0 deletions src/ebrains/+ebrains/+bucket/copyObject.m
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
function copyObject(bucketName, objectName, targetName, options)
% copyObject - Copy an object (file) of a Data Proxy bucket to another name
%
% Syntax:
% ebrains.bucket.copyObject(bucketName, objectName, targetName)
% writes a copy of the object under the name targetName, in the same
% bucket, and leaves the object itself in place. The Data Proxy
% copies the object within the object store, so no data passes
% through MATLAB. Both names are relative to the root of the bucket;
% a leading "/" is ignored. To copy a folder with everything in it,
% end both names with "/".
%
% The Data Proxy queues the copy and completes it in the background,
% and reports a copy that fails only by mail. So before the request,
% the function checks that the object is in the bucket (error
% EBRAINS:Bucket:ObjectNotFound otherwise) and, unless Overwrite is
% true, that no object has the target name (error
% EBRAINS:Bucket:TargetExists otherwise). A folder is present when
% any object is named below it. Two equal names, a folder name
% paired with an object name, and a target within the source folder
% are refused as well. No check sends the copy, so a refused copy
% produces no mail. The copy appears in the listing once it has
% completed.
%
% ebrains.bucket.copyObject(..., Name=Value) sets the options below.
%
% Input Arguments
% bucketName : Name of the bucket that holds the object
% objectName : Name of the object to copy
% targetName : Name of the copy
%
% Name-Value Arguments
% Overwrite : Replace an object that has the target name
% rather than refuse the copy. Default is false.
% SendSuccessEmail : Let the Data Proxy mail the account once the
% copy is done. Default is false; a copy that
% fails is reported by mail either way.
% Client : ebrains.bucket.api.BucketsClient that sends the
% requests. Meant for tests and custom clients; a
% default client is created otherwise.
%
% See also ebrains.bucket.renameObject, ebrains.bucket.deleteObject

arguments
bucketName (1,1) string
objectName (1,1) string
targetName (1,1) string
options.Overwrite (1,1) logical = false
options.SendSuccessEmail (1,1) logical = false
options.Client (1,1) ebrains.bucket.api.BucketsClient = ebrains.bucket.api.BucketsClient()
end

% Names are relative to the bucket root, so a leading "/" would be
% sent as an empty first folder.
objectName = ebrains.bucket.internal.removeLeadingSlash(objectName);
targetName = ebrains.bucket.internal.removeLeadingSlash(targetName);

ebrains.bucket.internal.checkSourceAndTarget(options.Client, bucketName, objectName, targetName, ...
options.Overwrite);

options.Client.copyObject(bucketName, objectName, targetName, ...
SendSuccessEmail=options.SendSuccessEmail);
end
42 changes: 36 additions & 6 deletions src/ebrains/+ebrains/+bucket/renameObject.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,49 @@ function renameObject(bucketName, objectName, targetName, options)
%
% Syntax:
% ebrains.bucket.renameObject(bucketName, objectName, targetName)
% gives the object the name targetName. Both names are relative to
% the root of the bucket; a leading "/" is ignored.
% gives the object the name targetName. The Data Proxy moves the
% object within the object store, so no data passes through MATLAB.
% Both names are relative to the root of the bucket; a leading "/"
% is ignored. To rename a folder with everything in it, end
% objectName with "/".
%
% The Data Proxy queues the rename and completes it in the
% background, and reports a rename that fails only by mail. So
% before the request, the function checks that the object is in the
% bucket (error EBRAINS:Bucket:ObjectNotFound otherwise) and, unless
% Overwrite is true, that no object has the target name (error
% EBRAINS:Bucket:TargetExists otherwise). A folder is present when
% any object is named below it. Two equal names, a folder name
% paired with an object name, and a target within the source folder
% are refused as well. No check sends the rename, so a refused
% rename produces no mail. The object can be listed under both
% names until the rename has completed.
%
% ebrains.bucket.renameObject(..., Name=Value) sets the options below.
%
% Input Arguments
% bucketName : Name of the bucket that holds the object
% objectName : Current name of the object
% targetName : New name of the object
%
% Name-Value Arguments
% Client : ebrains.bucket.api.BucketsClient that sends the request.
% Meant for tests and custom clients; a default client is
% created otherwise.
% Overwrite : Replace an object that has the target name
% rather than refuse the rename. Default is false.
% SendSuccessEmail : Let the Data Proxy mail the account once the
% rename is done. Default is false; a rename
% that fails is reported by mail either way.
% Client : ebrains.bucket.api.BucketsClient that sends the
% requests. Meant for tests and custom clients; a
% default client is created otherwise.
%
% See also ebrains.bucket.copyObject, ebrains.bucket.deleteObject

arguments
bucketName (1,1) string
objectName (1,1) string
targetName (1,1) string
options.Overwrite (1,1) logical = false
options.SendSuccessEmail (1,1) logical = false
options.Client (1,1) ebrains.bucket.api.BucketsClient = ebrains.bucket.api.BucketsClient()
end

Expand All @@ -28,5 +54,9 @@ function renameObject(bucketName, objectName, targetName, options)
objectName = ebrains.bucket.internal.removeLeadingSlash(objectName);
targetName = ebrains.bucket.internal.removeLeadingSlash(targetName);

options.Client.renameObject(bucketName, objectName, targetName);
ebrains.bucket.internal.checkSourceAndTarget(options.Client, bucketName, objectName, targetName, ...
options.Overwrite);

options.Client.renameObject(bucketName, objectName, targetName, ...
SendSuccessEmail=options.SendSuccessEmail);
end
Loading
Loading