When you want to expose files stored on an Azure Storage Account, you typically don’t want to grant access to the entire container.
One of the options is to generate a Secure Access Signature (SAS) which grants access to a specific blob for a certain amount of time. But, you don’t want to be generating these yourselves. So, why not use Azure API Management to host - and secure - an API which can generate these tokens upon request.
Steps
Overview
Let’s start with a high-level overview of how such a setup would look like and how this affects the flow of requests/responses.
As you can see, by using API Management you will be in full control of how clients have to authenticate/authorize, to ensure SAS-tokens are only being granted to those who deserve it. While this requires some additional setup from your end, no one will need to bother you anymore for generating a new SAS token to access one of your files.
Automate all the things!
Create a new API
As you can see in the overview, the API that is going to be used in this setup does not have any real backend. This means all of the magic will be done from within the policies, without requiring you to manage an additional app service/container/…
To give you a complete overview of how this type of API can be created, we’ll go step-by-step through the entire process.
To get started, we’ve created a new blank API within our API Management instance, specifically to host this storage endpoint.
Create a new operation
Now we have an API, it’s time to add the /generate/sas/{containerName}/{blobName}-endpoint, as this specific endpoint will grant you access to a specific file/blob.
Since the client will need to be able to provide the name of the blob and the name of the container in which the blob is located, the following template parameters have to be added.
While we’re at it, let’s also set the possible response-codes along with a sample response:
- 200 OK
- 401 Unauthorized
Once saved, the new endpoint has been created and is ready to be consumed, however as long as we haven’t specified any policies, it will not return anything useful.
Set the policy
Now that the infrastructure has been set, it’s time to dive into the policy.
As you will notice, we’re going to be using some inline C# here and there to create the signatures, which make up part of the SAS token.
Before you go to the policy editor, we will need to store the following values in a Named Value within API Management:
- KB-Storage-Account:
The name of the storage account where the blobs are located. - KB-Storage-Account-AccessKey:
An access key to be used within the creation of the signature.
Let’s go through each of the steps that are going to be performed by the policy, in order to make it clear what everything is doing.
-
Extract the parameters from the request:
<!-- extract parameters from URL --> <set-variable name="containerName" value="@(context.Request.MatchedParameters["containerName"])" /> <set-variable name="blobName" value="@(context.Request.MatchedParameters["blobName"])" />
-
Next, we need to initialize a set of variables that will be used for the creation of the SAS token.
You will notice that next to the expiration time (in seconds) we will also set the protocol to be used, the version of the storage account API, as well as the full name of the blob.
Some properties, such as signedIp, signedIdentifier, rscc (Cache-Control), rscd (Content-Disposition), rsce (Content-Encoding), rscl (Content-Language) and rsct (Content-Type) will remain empty in our example but could easily be provided for different scenarios.
More details on these properties can be found here.<!-- Initialize context variables with property values. --> <set-variable name="accessKey" value="{{KB-Storage-Account-AccessKey}}" /> <set-variable name="expiryInSeconds" value="3600" /> <set-variable name="storageAccount" value="{{KB-Storage-Account}}" /> <set-variable name="x-ms-date" value="@(DateTime.UtcNow)" /> <set-variable name="signedPermissions" value="r" /> <set-variable name="signedService" value="b" /> <set-variable name="signedStart" value="@(((DateTime)context.Variables["x-ms-date"]).ToString("yyyy-MM-ddTHH:mm:ssZ"))" /> <set-variable name="signedExpiry" value="@(((DateTime)context.Variables["x-ms-date"]).AddSeconds(Convert.ToInt32((string)context.Variables["expiryInSeconds"])).ToString("yyyy-MM-ddTHH:mm:ssZ"))" /> <set-variable name="signedProtocol" value="https" /> <set-variable name="signedVersion" value="2017-07-29" /> <set-variable name="canonicalizedResource" value="@{ return string.Format("/blob/{0}/{1}/{2}", (string)context.Variables["storageAccount"], (string)context.Variables["containerName"], (string)context.Variables["blobName"] ); }" /> <set-variable name="signedIP" value="" /> <set-variable name="signedIdentifier" value="" /> <set-variable name="rscc" value="" /> <set-variable name="rscd" value="" /> <set-variable name="rsce" value="" /> <set-variable name="rscl" value="" /> <set-variable name="rsct" value="" />
-
Combine all of the earlier defined properties into a string that will be used to create the signature.
<!-- Build the string to form signature. --> <set-variable name="StringToSign" value="@{ return string.Format("{0}\n{1}\n{2}\n{3}\n{4}\n{5}\n{6}\n{7}\n{8}\n{9}\n{10}\n{11}\n{12}", (string)context.Variables["signedPermissions"], (string)context.Variables["signedStart"], (string)context.Variables["signedExpiry"], (string)context.Variables["canonicalizedResource"], (string)context.Variables["signedIdentifier"], (string)context.Variables["signedIP"], (string)context.Variables["signedProtocol"], (string)context.Variables["signedVersion"], (string)context.Variables["rscc"], (string)context.Variables["rscd"], (string)context.Variables["rsce"], (string)context.Variables["rscl"], (string)context.Variables["rsct"] ); }" />
-
Now, we need to hash this string to obtain the actual signature.
<!-- Build/Hash the signature. --> <set-variable name="Signature" value="@{ byte[] SignatureBytes = System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"]); System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["accessKey"])); return Convert.ToBase64String(hasher.ComputeHash(SignatureBytes)); }" />
-
The signature has been created, so now we need to build up the entire SAS-token.
<!-- Form the sasToken. --> <set-variable name="SasToken" value="@{ return string.Format("sv={0}&sr={1}&sp={2}&st={3}&se={4}&spr={5}&sig={6}", (string)context.Variables["signedVersion"], (string)context.Variables["signedService"], (string)context.Variables["signedPermissions"], System.Net.WebUtility.HtmlEncode((string)context.Variables["signedStart"]), System.Net.WebUtility.HtmlEncode((string)context.Variables["signedExpiry"]), (string)context.Variables["signedProtocol"], System.Net.WebUtility.HtmlEncode((string)context.Variables["Signature"]) ); }" />
-
Before returning the SAS-token in itself, let’s build up the full URL so the client can immediately use this to download the file.
<!-- Form the complete Url. --> <set-variable name="FullUrl" value="@{ return string.Format("https://{0}.blob.core.windows.net/{1}/{2}?{3}", (string)context.Variables["storageAccount"], (string)context.Variables["containerName"], (string)context.Variables["blobName"], (string)context.Variables["SasToken"] ).Replace("+", "%2B"); }" />
-
But wait!
We’ve just generated a SAS-token, without verifying the existence of the file.
Let’s attempt to retrieve the metadata of the file, to verify it exists.<set-variable name="metadataUrl" value="@{ return string.Format("https://{0}.blob.core.windows.net/{1}/{2}?comp=metadata&{3}", (string)context.Variables["storageAccount"], (string)context.Variables["containerName"], (string)context.Variables["blobName"], (string)context.Variables["SasToken"] ).Replace("+", "%2B"); }" /> <!-- Check existance of the blob before returning 200 OK with sas token. --> <send-request mode="new" response-variable-name="blobMetadata" timeout="30" ignore-error="false"> <set-url>@((string)context.Variables["metadataUrl"])</set-url> <set-method>GET</set-method> </send-request>
-
Based on the response of the metadata call, return the SAS token or return a “NotFound”-exception.
<choose> <!-- Check active property in response --> <when condition="@(((IResponse)context.Variables["blobMetadata"]).StatusCode == 404)"> <!-- Return 404 Not Found with http-problem payload --> <return-response> <set-status code="404" reason="Not Found" /> <set-header name="Content-Type" exists-action="override"> <value>application/json</value> </set-header> <set-body>@{ return new JObject( new JProperty("error", "not_found"), new JProperty("error_description", "No data could be found for the given parameters."), new JProperty("timestamp", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ssZ")) ).ToString(); }</set-body> </return-response> </when> <otherwise> <!-- return response --> <return-response> <set-status code="200" reason="OK" /> <set-header name="Content-Type" exists-action="override"> <value>application/json</value> </set-header> <set-body>@{ var fullUrl = ((string)context.Variables["FullUrl"]); var remainingValidDuration = ((Convert.ToDateTime((string)context.Variables["signedExpiry"])) - DateTime.Now).TotalSeconds; return new JObject( new JProperty("url", fullUrl), new JProperty("expiresIn", Math.Floor(remainingValidDuration).ToString()), new JProperty("timestamp", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ssZ")) ).ToString(); }</set-body> </return-response> </otherwise> </choose>
If you want to see the entire policy, instead of the snippets, scroll down or click here.
Try it out!
Once everything has been saved, we can use Postman to build the request and retrieve a token.
If we first provide a wrong file name, we should now receive a “404 Not Found”-exception.
If we now provide a valid file name, we should receive a “200 OK”-response, along with the URL grant us access to the file.
Conclusion
By making use of API Management and the policy capabilities it is possible to create an API that allows clients to request SAS-tokens on the fly.
Next to saving you some time as you don’t have to generate these tokens yourself, you will also get additional logging within Application Insights, allowing you to see who is accessing what.
Full Policy
The full policy will look like this:
<policies>
<inbound>
<base />
<!-- extract parameters from URL -->
<set-variable name="containerName" value="@(context.Request.MatchedParameters["containerName"])" />
<set-variable name="blobName" value="@(context.Request.MatchedParameters["blobName"])" />
<!-- create SAS-token to access data on blob storage -->
<!-- Initialize context variables with property values. -->
<set-variable name="accessKey" value="{{KB-Storage-Account-AccessKey}}" />
<set-variable name="expiryInSeconds" value="3600" />
<set-variable name="storageAccount" value="{{KB-Storage-Account}}" />
<set-variable name="x-ms-date" value="@(DateTime.UtcNow)" />
<set-variable name="signedPermissions" value="r" />
<set-variable name="signedService" value="b" />
<set-variable name="signedStart" value="@(((DateTime)context.Variables["x-ms-date"]).ToString("yyyy-MM-ddTHH:mm:ssZ"))" />
<set-variable name="signedExpiry" value="@(((DateTime)context.Variables["x-ms-date"]).AddSeconds(Convert.ToInt32((string)context.Variables["expiryInSeconds"])).ToString("yyyy-MM-ddTHH:mm:ssZ"))" />
<set-variable name="signedProtocol" value="https" />
<set-variable name="signedVersion" value="2017-07-29" />
<set-variable name="canonicalizedResource" value="@{
return string.Format("/blob/{0}/{1}/{2}",
(string)context.Variables["storageAccount"],
(string)context.Variables["containerName"],
(string)context.Variables["blobName"]
);
}" />
<set-variable name="signedIP" value="" />
<set-variable name="signedIdentifier" value="" />
<set-variable name="rscc" value="" />
<set-variable name="rscd" value="" />
<set-variable name="rsce" value="" />
<set-variable name="rscl" value="" />
<set-variable name="rsct" value="" />
<!-- Build the string to form signature. -->
<set-variable name="StringToSign" value="@{
return string.Format("{0}\n{1}\n{2}\n{3}\n{4}\n{5}\n{6}\n{7}\n{8}\n{9}\n{10}\n{11}\n{12}",
(string)context.Variables["signedPermissions"],
(string)context.Variables["signedStart"],
(string)context.Variables["signedExpiry"],
(string)context.Variables["canonicalizedResource"],
(string)context.Variables["signedIdentifier"],
(string)context.Variables["signedIP"],
(string)context.Variables["signedProtocol"],
(string)context.Variables["signedVersion"],
(string)context.Variables["rscc"],
(string)context.Variables["rscd"],
(string)context.Variables["rsce"],
(string)context.Variables["rscl"],
(string)context.Variables["rsct"]
);
}" />
<!-- Build/Hash the signature. -->
<set-variable name="Signature" value="@{
byte[] SignatureBytes = System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"]);
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["accessKey"]));
return Convert.ToBase64String(hasher.ComputeHash(SignatureBytes));
}" />
<!-- Form the sasToken. -->
<set-variable name="SasToken" value="@{
return string.Format("sv={0}&sr={1}&sp={2}&st={3}&se={4}&spr={5}&sig={6}",
(string)context.Variables["signedVersion"],
(string)context.Variables["signedService"],
(string)context.Variables["signedPermissions"],
System.Net.WebUtility.HtmlEncode((string)context.Variables["signedStart"]),
System.Net.WebUtility.HtmlEncode((string)context.Variables["signedExpiry"]),
(string)context.Variables["signedProtocol"],
System.Net.WebUtility.HtmlEncode((string)context.Variables["Signature"])
);
}" />
<!-- Form the complete Url. -->
<set-variable name="FullUrl" value="@{
return string.Format("https://{0}.blob.core.windows.net/{1}/{2}?{3}",
(string)context.Variables["storageAccount"],
(string)context.Variables["containerName"],
(string)context.Variables["blobName"],
(string)context.Variables["SasToken"]
).Replace("+", "%2B");
}" />
<set-variable name="metadataUrl" value="@{
return string.Format("https://{0}.blob.core.windows.net/{1}/{2}?comp=metadata&{3}",
(string)context.Variables["storageAccount"],
(string)context.Variables["containerName"],
(string)context.Variables["blobName"],
(string)context.Variables["SasToken"]
).Replace("+", "%2B");
}" />
<!-- Check existance of the blob before returning 200 OK with sas token. -->
<send-request mode="new" response-variable-name="blobMetadata" timeout="30" ignore-error="false">
<set-url>@((string)context.Variables["metadataUrl"])</set-url>
<set-method>GET</set-method>
</send-request>
<choose>
<!-- Check active property in response -->
<when condition="@(((IResponse)context.Variables["blobMetadata"]).StatusCode == 404)">
<!-- Return 404 Not Found with http-problem payload -->
<return-response>
<set-status code="404" reason="Not Found" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-body>@{
return new JObject(
new JProperty("error", "not_found"),
new JProperty("error_description", "No data could be found for the given parameters."),
new JProperty("timestamp", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ssZ"))
).ToString();
}</set-body>
</return-response>
</when>
<otherwise>
<!-- return response -->
<return-response>
<set-status code="200" reason="OK" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-body>@{
var fullUrl = ((string)context.Variables["FullUrl"]);
var remainingValidDuration = ((Convert.ToDateTime((string)context.Variables["signedExpiry"])) - DateTime.Now).TotalSeconds;
return new JObject(
new JProperty("url", fullUrl),
new JProperty("expiresIn", Math.Floor(remainingValidDuration).ToString()),
new JProperty("timestamp", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ssZ"))
).ToString();
}</set-body>
</return-response>
</otherwise>
</choose>
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>