Visualization of Risk Premium in Corn and Class III Milk Markets

May 2012 - Jun. 2012

This project was the first major project my adviser assigned to me, shortly after we had a brief introductory meeting with each other. In order to make a name for myself for the rest of my PhD career, I decided to deliver a top notch performance with a well-intended "wow" effect in mind. 

The goal of the project is to generate data / charts and look at how the futures risk premium looks like. 

Economic theory states that if a commodity futures market is efficient, futures price should always equal the expected spot price of the underlying commodity. When these two don't match, a so-called risk premium arises to compensate the risk of either buying or selling the futures, but not both. Depending on the sign of the difference, we can imply which side the market is rewarding. For example, if the futures price is higher than the expected spot price, by applying the buy-low-sell-high principle, one can deduce that the seller "benefits" from selling at the current futures price that is higher than its theoretical fair value. In other words, the seller demands a risk premium to take on the risk of pre-sell a commodity today. 

But how do you visualize risk premium? My adviser came up with this genius idea of looking at the percentage deviation of a futures price from its spot price over time for each fixed time-to-maturity value. Here's how the simple (I promise it's simple!) math looks like. 

Here, the epsilon on the left hand side of the equation denotes the percentage deviation my adviser termed Realized Prediction Error (RPE); F denotes futures price; P represents its corresponding spot price; t denotes the trading day; T represents the expiration date. As a technical footnote, I want to point out that an implicit assumption here is that RPE is ergodic, meaning if we do this over and over and get a bird's eye view of RPE, we get the actual distribution of the RPE. 

My adviser laid out the theoretical work. Now someone needs to do the actual visualization to see if the theory holds. Here is where my part comes in. 

I was given corn futures daily settlement prices from Jan. 3rd, 1983 to Feb. 25th, 2011 and Class III milk futures from Jan. 3rd, 2000 to May 18th, 2011. The corn prices are batched in 151 csv files. The Class III prices come in with 160 files. Each file contains prices of a particular maturity. 

First thing I had to do is getting those files into a manageable form. Going through text files line by line to pull data is simply not a good strategy especially when you have 311 of them. What I ended up doing was sorting through all files and stacking useful information into two files. One for each commodity. 

Here's how one of the raw data files looks like on Excel. 
The first column is the contract identifier used by the source. The second column is the trading day. The next four columns, C-F, are Open, High, Low, Close prices. Column G is supposed to be the trading volume. Column H is supposed to be Open interest. Our data source did not have inputs on those two data fields. Luckily they were not used in this project. 

Below is a screenshot of how the processed data look like. 
Column A - H as you can see, are direct copies of the raw file. Column I to M are calculated columns to facilitate the visualization of RPE's. Column L is the value of RPE. In the final charts which I will show below, the x-axis displays the value of column M and the y-axis displays the value of column L. In each processed file, contracts with the same ordinal month value (1=Jan, 2=Feb, etc) are stacked into a same sheet. 

Once all raw data files are reshaped into something like the image shown above, it is time to group them nicely for plotting purpose. A user interface is developed in a separate Excel file. As I mentioned at the beginning, this project is meant to "wow" my adviser, a lot of care was given to make sure everything runs robustly and smoothly. Here's how the user interface looks like:
As you can see, the control panel allows for a range of customization. You can pick the length of the time series, choose what's on the x-axis, exclude particular contract months and decide the content of a statistic table. To make sure a wow effect is ultimately delivered, the user interface even shows a progress window in modal state while it is generating results. I also made sure the entire file responds to the pressing of the escape key in case my adviser wants to abort the process mid way through. Here's how the window looks like:

After all this programming gimmicks, an Excel file is created in a different instance. The reason for hosting the result file in a separate process is to preserve half-churned results even if the main logic file -- the user interface file -- crashes. The result file is split into two sheets. The first sheet shows the RPE chart that visualizes the risk premium. The second sheet contains a table of statistics. 

Here is how the RPE chart for Class III milk looks like. Each gray dot on the chart represents a particular RPE value. 
The blue band in the chart represents the 10% confidence interval. As you can see from the statistical table below, it is constructed by trimming upper and lower 5% standard deviation from data points for each time to maturity. 

Here is how the corresponding statistical table looks like:

The economics of this project:
As we can see from the Class III chart, the dots are very dispersed at longer time horizons (higher time to maturity). As a point to note, when the series get to the contract expiration dates (equivalently that's when futures price converges to spot price in theory), Class III milk market can almost perfectly price the contracts. See the gray cone converges to 0% in the middle of the last 30 days and stays there until the end. In contrast, the corn contracts can only manage to stay at around -5% to +5% pricing error at when contracts expire. See chart below. 

What all these mean is that the market demands a higher risk premium in corn and Class III futures markets when those contracts are far away from expiration. It makes sense when contracts are so far from expiration dates, market participants feel less certain about where the prices will end up being and thus anticipate compensation for bearing such uncertainty / risk. Perhaps because corn is a storable commodity and corn futures is physically deliverable, the risk premium still exits on the day a contract expires. 

In case you are interested, here's a snippet of the source code written in the archaic VBA.