Metadata Search Operation

The metadata search feature expands on the GET Bucket S3 API. It allows users to conduct metadata searches by adding the custom ARTESCA Data Services querystring parameter, search. The search parameter is structured as a pseudo-SQL WHERE clause and supports basic SQL operators. For example, "A=1 AND B=2 OR C=3". More complex queries can also be made using nesting operators, “(” and “)“.

The search process is as follows:

1. ARTESCA Data Services receives a GET request containing a search parameter:

   GET /bucketname?search=key%3Dsearch-item HTTP/1.1
   Host: zenko.local:80
   Date: Wed, 18 Oct 2018 17:50:00 GMT
   Authorization: <authorization string>
   

   By default, the above request will only return the latest version of objects that meet the search query, including versions parameter in the request will list all satisfied versions.

   GET /bucketname?search=key%3Dsearch-item&&versions HTTP/1.1
   Host: zenko.local:80
   Date: Wed, 18 Oct 2018 17:50:00 GMT
   Authorization: <authorization string>
   

2. CloudServer parses and validates the search string:

   • If the search string is invalid, CloudServer returns an InvalidArgument error.

   • If the search string is valid, CloudServer parses it and generates an abstract syntax tree (AST).

3. CloudServer passes the AST to the MongoDB backend as the query filter for retrieving objects in a bucket that satisfies the requested search conditions.

4. CloudServer parses the filtered results and returns them as the response.

   Without versions parameter, search results are structured the same as GET bucket objects results:

   <?xml version="1.0" encoding="UTF-8"?>
   <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
       <Name>bucketname</Name>
       <Prefix/>
       <Marker/>
       <MaxKeys>1000</MaxKeys>
       <IsTruncated>false</IsTruncated>
       <Contents>
           <Key>objectKey</Key>
           <LastModified>2018-04-19T18:31:49.426Z</LastModified>
           <ETag>&quot;d41d8cd98f00b204e9800998ecf8427e&quot;</ETag>
           <Size>0</Size>
           <Owner>
               <ID>79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be</ID>
               <DisplayName>Bart</DisplayName>
           </Owner>
           <StorageClass>STANDARD</StorageClass>
       </Contents>
       <Contents>
           ...
       </Contents>
   </ListBucketResult>
   

   With versions parameter, search results are structured the same as GET bucket object versions results:

   <?xml version="1.0" encoding="UTF-8"?>
   <ListVersionsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
       <Name>bucketname</Name>
       <MaxKeys>1000</MaxKeys>
       <IsTruncated>false</IsTruncated>
       <Version>
           <Key>objectKey</Key>
           <VersionId>39383335323035333934353735313939393939395247303031202036383038312e32</VersionId>
           <IsLatest>true</IsLatest>
           <LastModified>2018-04-19T18:31:49.426Z</LastModified>
           <ETag>&quot;12ac9a26a65a57bacd9a15f64b8a29b1&quot;</ETag>
           <Size>11</Size>
           <Owner>
               <ID>79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be</ID>
               <DisplayName>Bart</DisplayName>
           </Owner>
           <StorageClass>STANDARD</StorageClass>
       </Version>
       <Version>
           <Key>objectKey</Key>
           <VersionId>39383335323035333934383937373939393939395247303031202036383038312e31</VersionId>
           <IsLatest>false</IsLatest>
           <LastModified>2018-04-19T18:30:49.426Z</LastModified>
           <ETag>&quot;12ac9a26a65a57bacd9a15f64b8a29b1&quot;</ETag>
           <Size>11</Size>
           <Owner>
               <ID>79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be</ID>
               <DisplayName>Bart</DisplayName>
           </Owner>
           <StorageClass>STANDARD</StorageClass>
       </Version>
       <Version>
           ...
       </Version>
   </ListVersionsResult>