Microsoft Azure Media Services – How to encode in the Azure Cloud – Part 2

2012-09-12T14:58:00+02:00

After the first post which give an overview and explain how set-up a WAMS environment, I continue the discovery of WAMS World (sorry for the pun ;-)). In this post, we will see how we can transform a video from a input format to another format. It's possible to generate multiple output format or to encrypt the video. But we will see it in a next posts.

First of all, we need to understand the different terms we will use in WAMS. In my example, I want to obtain this workflow :

create an asset : an asset is an entity which contains all informations on the video (metadata, ...)
upload a file in blob storage and associate to the asset
apply a job on the asset. A job is composed by one or more tasks. A task is an action : eg: encoding with protection. To run the task, we use a MediaProcessor.
after the job completed, deliver the video via the Azure CDN.

Create an asset

As I said previously, the AMS is based on REST, so you use HTTP verbs to make an action. Here is the PHP class, I used to make API call (I completed the code given in previous post):

[php]class Rest{
protected \$strUrl;
protected \$strToken;
protected \$strBody;
public function __construct(){
\$this->requestToken();
}

private function generateData(\$arrData){
return implode('&', \$arrData);
}

public function requestToken(){
if (file_exists(TOKEN_STORAGE)){
echo 'Get Token from storage'."\n";
\$data = file_get_contents(TOKEN_STORAGE);
\$arrToken = json_decode(\$data);
if ((filemtime(TOKEN_STORAGE) + \$arrToken->expires_in) > time()) {
\$this->strToken = \$arrToken->access_token;
} else {
echo 'Token expired'."\n";
unlink(TOKEN_STORAGE);
\$this->requestToken();
}
} else {
echo 'Get new token from API'."\n";
\$arrData = array(
'grant_type=client_credentials',
'client_id='.CLIENT_ID,
'client_secret='.urlencode(ACCESS_KEY),
'scope=urn%3aWindowsAzureMediaServices'
);
\$arrHeader = array(
'Content-length:'.strlen(\$this->generateData(\$arrData))
);

\$ch = curl_init();
curl_setopt(\$ch, CURLOPT_URL, TOKEN_URL);
curl_setopt(\$ch, CURLOPT_POSTFIELDS, \$this->generateData(\$arrData));
curl_setopt(\$ch, CURLOPT_HTTPHEADER, \$arrHeader);
curl_setopt(\$ch, CURLOPT_RETURNTRANSFER, true);
\$data = curl_exec(\$ch);
curl_close(\$ch);

\$arrToken = json_decode(\$data);
if (isset(\$arrToken->error)){
print_r(\$arrToken);
die();
}
\$this->strToken = \$arrToken->access_token;
file_put_contents(TOKEN_STORAGE, \$data);
echo 'Token save in storage'."\n";
}
return \$this->strToken;
}

public function request(\$arrData = array()){
\$ch = curl_init();
\$arrHeader = array(
'x-ms-version:1.0',
'DataServiceVersion:3.0',
'MaxDataServiceVersion:3.0',
'Authorization: Bearer '.\$this->strToken,
'Content-Type: application/json;odata=verbose',
'Accept: application/json;odata=verbose'
);
echo 'Call API:'.\$this->strUrl."\n";
curl_setopt(\$ch, CURLOPT_URL, \$this->strUrl);
curl_setopt(\$ch, CURLOPT_HTTPHEADER, \$arrHeader);
curl_setopt(\$ch, CURLOPT_HEADER, true);
curl_setopt(\$ch, CURLOPT_RETURNTRANSFER, true);
if (!empty(\$arrData)){
curl_setopt(\$ch, CURLOPT_POSTFIELDS, json_encode(\$arrData));
}
\$data = curl_exec(\$ch);
\$arrInfo = curl_getinfo(\$ch);
curl_close(\$ch);

list(\$strHeader, \$strBody) = explode("\r\n\r\n", \$data, 2);

if (\$arrInfo['http_code'] == 301){
if(preg_match('`\s*Location:\s(.+)`i', \$data, \$arrTmp)){
\$arrTmp[1] = trim(\$arrTmp[1]);
Register::getInstance()->setIndex('base_url', \$arrTmp[1]);
\$this->strUrl = \$arrTmp[1].\$this->getVerb();
echo 'Redirection to '.\$this->strUrl."\n";
\$this->request(\$arrData);
}
} else {
\$this->strBody = \$strBody;
}
}

private function getVerb(){
\$arrUrl = explode('/', \$this->strUrl);
return array_pop(\$arrUrl);
}
}[/php]

You can look, the code manages the token, so you only need to make the API call.

Now, we will create an asset. To do that, we need

an asset name; in my case : first_sintel_test.
the URI of the API : https://wamsbluclus001rest-hs.cloudapp.net/API/Assets

We will create an asset with POST verb.

[php]class Asset extends Rest{

public function __construct(){
parent::__construct();
}

public function createAsset(\$strName) {
\$strAPIname = 'Assets';
\$this->strUrl = 'https://wamsbluclus001rest-hs.cloudapp.net/API/'.\$strAPIname;
\$arrCreate = array('Name' => \$strName);
\$this->request(\$arrCreate);
}
}

\$obj = new Asset();
\$str = \$obj->createAsset('first_sintel_test');

print_r(json_decode(\$str));
[/php]

Ok, the server answers, the asset was created. Now, we will verified by listing the different asset. The method below will be added to the previous one.

[php]public function listAsset() {
\$strAPIname = 'Assets';
\$this->strUrl = Register::getInstance()->getIndex('base_url').\$strAPIname;
\$this->request();
return \$this->strBody;
}
[/php]

Note the asset ID, you need it after. Now, we call the list Asset API.

[php]
\$obj = new Asset();
\$str = \$obj->listAsset();

print_r(json_decode(\$str));
[/php]

Good, we have our first_sintel_test asset with his creation date, his status and some others properties. You can filter to get informations about one asset by adding the ID (a unique identifier).

Now, we can upload our file in blob storage.

Upload a file and associate to asset

To upload a file and associate to an asset, you need to give access via policies (time duration and permission: read, write, delete,...). You can create the policy, the process is the same like the Asset creation. Note the access ID, you need it after.

[php]
\$obj = new AccessPolicy();
\$str = \$obj->createAssetPolicy('sintel_policy', 30, 2);
print_r(json_decode(\$str));
[/php]

We have now a policy to upload the file, we have to get a URL path to upload by created a Locator. When it will be created, it will available in the Asset list API. To create a Locator, you need 4 informations :

the asset ID (you noted it before)
the access policy ID (noted it too)
the start time : this one is to give the access in the future or now (have a look on documentation's tips to give access immediately).
the type of access (SAS, Origin or Azure CDN)

Ok, now we can upload the file with Windows Azure Storage Services REST API. You can check that your video was uploaded correctly via the Azure Portal : Storage -> Select your storage -> Containers and you can see it :

Now, before process the video we need to publish the asset. Take care on the Content-length header's value, it needs to be set at 0. I make a little update on the code.

[php]
public function request(\$arrData = array(), \$strTypePost = false){
....
echo 'Call API:'.\$this->strUrl."\n";
if (\$strTypePost && empty(\$arrData)) {
curl_setopt(\$ch, CURLOPT_CUSTOMREQUEST, 'POST');
\$arrHeader[] = 'Content-Length: 0';
}
curl_setopt(\$ch, CURLOPT_URL, \$this->strUrl);
....
\$this->strUrl = \$arrTmp[1].\$this->getVerb();
echo 'Redirection to '.\$this->strUrl."\n";
\$this->request(\$arrData, \$strTypePost);
....
}[/php]

Then, you will receive a 204 header code.

Create a job

Ok, now, we have the asset and the file. But, in which format do we want to transcode ? Microsoft give us a list of presets to make Task on WAMS : transcode in differents commons formats (for example iOS), apply protection (via PlayReady DRM) or rewrap contents from Smooth to HLS. I choose this preset : H.264 iPod Classic / Nano. We can apply multiple tasks on an asset, but we will this it in a next post.

Let's go to create our first job and apply a task preset.

[php]class Jobs extends Rest{

public function __construct(){
\$this->requestToken();
}

public function listJobs(){
\$strAPIname = 'Jobs';
\$this->strUrl = Register::getInstance()->getIndex('base_url').\$strAPIname;
\$this->request();
return \$this->strBody;
}

public function createJob(\$strName, \$strAssetUri, \$strConfig, \$strProcessor, \$strTaskBody) {
\$strAPIname = 'Jobs';
\$this->strUrl = Register::getInstance()->getIndex('base_url').\$strAPIname;

\$objMedia = new stdClass();
\$objMedia->__metadata->uri = \$strAssetUri;

\$objTask = new stdClass();
\$objTask->Configuration = \$strConfig;
\$objTask->MediaProcessorId = \$strProcessor;
\$objTask->TaskBody = \$strTaskBody;

\$arrCreate = new stdClass();
\$arrCreate->Name = \$strName;
\$arrCreate->InputMediaAssets[] = \$objMedia;
\$arrCreate->Tasks[] = \$objTask;

\$this->request(\$arrCreate);
return \$this->strBody;
}

public function cancelJob(\$assetId) {
\$strAPIname = 'CancelJob?jobid=';

\$this->strUrl = Register::getInstance()->getIndex('base_url').\$strAPIname."'".urlencode(\$assetId)."'";

\$this->request();
return \$this->strBody;
}
}[/php]

This class provides methods to list, create and cancel a job. Here is a API creation call :

[php]
\$objJob = new Jobs();
\$str = \$objJob->createJob('EncoreSecondSintel', \$assetUri, \$strConf, \$strProcess, \$strTaskBody);
print_r(json_decode(\$str));
[/php]

\$assetUri is given when you create the asset, \$strConf is the preset we choose (the string H.264 iPod Classic / Nano), \$strTaskBody is a XML string :

[xml]\<?xml version="1.0" encoding="utf-16"?>
\<taskBody>
\<inputAsset>JobInputAsset(0)\</inputAsset>
\<outputAsset>JobOutputAsset(0)\</outputAsset>
\</taskBody>[/xml]

There is a last parameter : \$strProcess. This is the ID of the MediaProcessor we want to use. There is 4 types of MediaProcessors :

Storage Decryption
Windows Azure Media Encoder
MP4 to Smooth Streams Task
PlayReady Protection Task
Smooth Streams to HLS Task

In our case, we are using the Windows Azure Media Encoder, so apply the associated ID give by the MediaProcessor API on the \$strProcess param.

Ok, now we can create a Job. The Job entity have different properties :

Job ID
Time datas (creation, end of task, running duration,...)
State
Tasks details. This parameter provide us some additionnals informations like the performance :

EncodeTime: 00:00:20.9047240
Avg CPU Load: 70.3% Avg Active Core: 3.9/4
Download: 00:00:02.1264505
Upload: 00:00:09.4015013

Great, the job is running, we can follow the progression with the listJobs() method.

When the job is done, the movie is on blob storage in Azure. There is few solutions :

download the movie
publish the movie on the Azure CDN (for streaming)

In our case, we will download the movie with Windows Azure Storage Services REST API. After few seconds (or minutes it depends on your movie size), you will have the video on your local storage and view it.

To finish :

the transfert (upload/download : on/from Azure and from AzureStorage to MediaService instances) on the differents files seems to take more time than the transcoding process
for a good usage, you need to understand all the entities available and how they will be implemented
there is many possibilities, so don't hesitate to test the multiple combinaison you think. I will make other(s) post(s) on this subject.

Microsoft Azure Media Services - How to encode in the Azure Cloud - Part 1

2012-08-24T15:25:00+02:00

I open a serie of posts which will talk about this solution. Microsoft announced at the end of spring the preview of Windows Azure Media Services. WAMS form an extensible media platform that integrates the best of the Microsoft Media Platform and third-party media components in Windows Azure.

Overview

WAMS can transform movie in differents formats and codecs :

Supported video and audio codecs:
H.264 High, Main and Baseline Profiles
AAC-LC
HE-AAC
VC-1 (Windows Media Video 9) Simple, Main and Advanced Profiles
Windows Media Audio Standard
Windows Media Audio Professional
Supported format conversions:
ISO MP4 (.mp4) to Smooth Streaming File Format (PIFF 1.3) (.ismv; .isma)
Smooth Streaming File Format (PIFF) to Apple HTTP Live Streaming (.msu8, .ts)

For the moment, you can only make VoD video.
It is already possible to encrypt your content (via PlayReady DRM). You can manage it via .NET or REST API. I'm not a .NET developer, so I will try the solution via API. My code side will be in PHP.

Setup

First of all, you need to setup your account to use WAMS. Currently, the setup is only possible via Powershell, but it will change via the Azure portal. For this step, you need a Windows, but after you can use other OS.

To connect to Media services, you need two things : 1) an access token provided by Windows Azure Access Control Services (ACS) and 2) the URI of Media Services itself. To get the token, you make a OAuth request with the parameter you get when you setup your account : the access key and the client ID. Different libraries are available in different langage to make this kind of request, for me, I create a simple PHP method to manage it.

[php]class Rest{

    protected $strUrl;

    protected $strToken;

    protected $strBody;

    public function requestToken(){
        if (!file_exists(TOKEN_STORAGE)){
            echo 'Get Token from storage'."\n";
            $data = file_get_contents(TOKEN_STORAGE);
            $arrToken = json_decode($data);
            if ((filemtime(TOKEN_STORAGE) + $arrToken-&gt;expires_in) &gt; time()) {
                $this-&gt;strToken = $arrToken-&gt;access_token;
            } else {
                echo 'Token expired'."\n";
                unlink(TOKEN_STORAGE);
                $this-&gt;requestToken();
            }
        } else {
            echo 'Get new token from API'."\n";
            $arrData = array(
                'grant_type=client_credentials',
                'client_id='.CLIENT_ID,
                'client_secret='.urlencode(ACCESS_KEY),
                'scope=urn%3aWindowsAzureMediaServices'
            );
            $arrHeader = array(
                'Content-length:'.strlen($this-&gt;generateData($arrData))
            );

            $ch = curl_init();
            curl_setopt($ch, CURLOPT_URL, TOKEN_URL);
            curl_setopt($ch, CURLOPT_POSTFIELDS, $this-&gt;generateData($arrData));
            curl_setopt($ch, CURLOPT_HTTPHEADER, $arrHeader);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            $data = curl_exec($ch);
            curl_close($ch);

            $arrToken = json_decode($data);
            if (isset($arrToken-&gt;error)){
                print_r($arrToken);
                die();
            }
            $this-&gt;strToken = $arrToken-&gt;access_token;
            file_put_contents(TOKEN_STORAGE, $data);
            echo 'Token save in storage'."\n";
        }
         return $this-&gt;strToken;
    }
}[/php]

The AMS response will contains the token. If you make many calls, I suggest to cache the access token, the expires_in value and the timestamp of the request to avoid unnecessary calls.

Now, we have a token to access, we can play with the Azure Media Services. We will see it in next posts.