Solving a simple use case with Azure Functions – Part 1

Ok, so let’s have some fun with some code in the cloud tonight. I will take a simple use case and I will implement a solution with Azure Functions, just to show how easy and versatile this is.

The use case will be to create a custom web page that will present the latest price for a stock. The needed information will be extracted from a public Finance Information API.

However, what I will do, can easily be generalized and applied to any public or private API, the extracted information can be stored in a database instead of being displayed in a web page, etc.

The workflow

First, let’s lay out the desired workflow:

When I want to know the latest stock price, I will open a specific URL with a browser
At the backend, the public Finance Information API will be called and the information will be extracted
The formatted information will be presented at the web page

This is the simplest workflow to start with. I might improve it later with caching, preemptive lookup, etc.

The output

Let’s start with what I would like to see in my custom page.

To keep things simple, I will display the latest price of the chosen company’s stock, the change percentage from the previous close and the date/time that corresponds to this price. Something like that:

MSFT: $255.02 (+0.13%) 02/12/2022 21:00:00 GMT

I chose the symbol MSFT, that corresponds to Microsoft’s stock, for this lab.

Analyze the source API

The Finance Information API that I will use is Yahoo! Finance, because it’s simple and free.

It is possible to ask for multiple stock prices at the same time, but for this lab, I will use a simple quote query, like so:

https://query1.finance.yahoo.com/v7/finance/quote?symbols=MSFT

The result comes in JSON format and contains lots of information for the stock (formatted for clarity here):

{
  "quoteResponse": {
    "result": [
      {
        "language": "en-US",
        "region": "US",
        "quoteType": "EQUITY",
        "typeDisp": "Equity",
        "quoteSourceName": "Nasdaq Real Time Price",
        "triggerable": true,
        "customPriceAlertConfidence": "HIGH",
        "currency": "USD",
        "marketState": "CLOSED",
        "firstTradeDateMilliseconds": 511108200000,
        "priceHint": 2,
        "postMarketChangePercent": -0.0862682,
        "postMarketTime": 1670029189,
        "postMarketPrice": 254.8,
        "postMarketChange": -0.220001,
        "regularMarketChange": 0.33000183,
        "regularMarketTime": 1670014804,
        "regularMarketDayHigh": 256.05,
        "regularMarketDayRange": "249.75 - 256.05",
        "regularMarketDayLow": 249.75,
        "regularMarketVolume": 21528520,
        "financialCurrency": "USD",
        "exchange": "NMS",
        "shortName": "Microsoft Corporation",
        "longName": "Microsoft Corporation",
        "messageBoardId": "finmb_21835",
        "exchangeTimezoneName": "America/New_York",
        "exchangeTimezoneShortName": "EST",
        "gmtOffSetMilliseconds": -18000000,
        "market": "us_market",
        "esgPopulated": false,
        "regularMarketChangePercent": 0.12956999,
        "regularMarketPrice": 255.02,
        "epsCurrentYear": 9.55,
        "twoHundredDayAverageChangePercent": -0.03831444,
        "priceEpsCurrentYear": 26.703665,
        "sharesOutstanding": 7454470144,
        "bookValue": 23.276,
        "fiftyDayAverage": 237.8712,
        "fiftyDayAverageChange": 17.148804,
        "fiftyDayAverageChangePercent": 0.07209281,
        "twoHundredDayAverage": 265.18024,
        "twoHundredDayAverageChange": -10.160233,
        "marketCap": 1901039058944,
        "forwardPE": 22.810375,
        "priceToBook": 10.95635,
        "sourceInterval": 15,
        "exchangeDataDelayedBy": 0,
        "averageAnalystRating": "1.8 - Buy",
        "tradeable": false,
        "cryptoTradeable": false,
        "regularMarketPreviousClose": 254.69,
        "bid": 254.8,
        "ask": 254.94,
        "bidSize": 10,
        "askSize": 14,
        "fullExchangeName": "NasdaqGS",
        "regularMarketOpen": 249.82,
        "averageDailyVolume3Month": 29578579,
        "averageDailyVolume10Day": 24021410,
        "earningsTimestamp": 1666733400,
        "earningsTimestampStart": 1674471540,
        "earningsTimestampEnd": 1674820800,
        "trailingAnnualDividendRate": 2.54,
        "trailingPE": 27.451023,
        "trailingAnnualDividendYield": 0.009972908,
        "epsTrailingTwelveMonths": 9.29,
        "epsForward": 11.18,
        "fiftyTwoWeekLowChange": 41.59001,
        "fiftyTwoWeekLowChangePercent": 0.19486488,
        "fiftyTwoWeekRange": "213.43 - 344.3",
        "fiftyTwoWeekHighChange": -89.27998,
        "fiftyTwoWeekHighChangePercent": -0.2593087,
        "fiftyTwoWeekLow": 213.43,
        "fiftyTwoWeekHigh": 344.3,
        "dividendDate": 1678320000,
        "displayName": "Microsoft",
        "symbol": "MSFT"
      }
    ],
    "error": null
  }
}

After looking at the data and some investigation, I see that in order to display the information I need, I have to extract the values of the following fields:

Information	Field
latest price	`regularMarketPrice`
change from previous close	`regularMarketChangePercent`
date/time of latest price	`regularMarketTime`

The date/time is given in Unix timestamp format, so I will have to convert it to human readable form.

How to read and parse the data

Reading and parsing the json data seems like an easy task. It will be easier doing it in a programming language that I am familiar with. At the time of this writing, the following languages (and versions) can be used with Azure Functions:

Language	Runtime 4.x
C#	.NET 6.0/7.0 and .NET Framework 4.8
F#	.NET 6.0/7.0
Java	Java 11 & 8
Python	3.7, 3.8, 3.9
PowerShell	7.0, 7.2
JavaScript	Node.js 14/16
TypeScript	(supported through transpiling to JavaScript)

Notes:

Only Generally Available (fully supported and approved for production use) options are listed
Only the latest runtime is listed

I will use Python for this lab, but I might revisit it again in the future and try the implementation with the other options (maybe compare them as well).

Create a project to work with

First, I will create a simple project, to write and test some code and make sure that I have the basic functionality in place, before moving on to Azure Function specifics.

A Linux environment will be used for this lab, but the workflow is similar in a windows environment as well.

I will start by setting up a GitHub repository tlnl-001-azfunc-py based on a Python template and then, I will clone the GitHub repository to my workstation to start working:

git clone git@github.com:gpap/tlnl-001-azfunc-py.git

Then I will setup the python virtual environment and activate it:

cd tlnl-001-azfunc-py
python3 -m venv .venv --prompt "tlnl-001-azfunc-py"
source .venv/bin/activate

After that, I can fire up Visual Studio Code by:

code .

Make a prototype

Let’s put the JSON output from the finance API into a file named test_api_sample_msft.json, to start prototyping my code:

curl https://query1.finance.yahoo.com/v7/finance/quote?symbols=MSFT > api_output_sample_msft.json

Or, you can just copy and paste the output you got before into the file manually, if you’re that kind of person 🙂

Let’s write some code to extract the information that I want from this JSON file. What I need is to:

read the contents of the file
parse the data and find the information I need
display the information in the desired format

It’s always a good practice to write tests for your code. In fact, it’s even better to write your tests before writing the code to be tested (that’s a development style known as Test Driven Development). So, I’ll start with some simple tests in a file test.py:

import unittest
import stock  # our code will be here

class TestStock(unittest.TestCase):

  # test parameters and expected data
  params = {
    'data_filename' : 'test_api_sample_msft.json',
    'data' : {
      'symbol' : 'MSFT',
      'regularMarketPrice' : 255.02,
      'regularMarketChangePercent' : 0.12956999,
      'regularMarketTime' : 1670014804
    }
  }

  # test if input contains the expected fields and we were able to read them
  def test_read_file(self):
    data_filename = self.params['data_filename']
    data_expected = self.params['data']
    data = stock.get_stock_data_from_file(data_filename, data_expected.keys())
    for field, value in data_expected.items():
      self.assertIn(field, data)

  # test if the returned data values are the expected ones
  def test_data(self):
    data_filename = self.params['data_filename']
    data_expected = self.params['data']
    data = stock.get_stock_data_from_file(data_filename, data_expected.keys())
    for field, value in data_expected.items():
      self.assertEqual(data[field], value)


if __name__ == "__main__":
  unittest.main()

And let’s write the simplest code to pass my tests in a file stock.py:

import json

# read from a filename containing Yahoo Finance stock data,
# find specified keys and return their values in a dictionary
def get_stock_data_from_file(filename, keys = None):
  if not keys:
    keys = [ 'symbol', 'regularMarketPrice', 'regularMarketChangePercent', 'regularMarketTime' ]
  try:
    with open(filename, 'r') as f:
      data_in = json.load(f)
      data_out = {}
      for key in keys:
        data_out[key] = data_in['quoteResponse']['result'][0][key]
      return data_out
  except Exception as e:
    print("error getting data from file: {}".format(repr(e)))
    return None

Let’s run the test:

python test.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

More prototyping – display data in simple text and HTML

Now, I will add some code that will display the data on the console and also format the data as simple HTML.

Writing tests for the output might seem like an overkill at this point, but it’s a good habit to get into. You don’t want to end up with code that does all the business logic right but fails to display the desired data.

Having said that, care should be taken to test just for the presence of important information and not for superficial details.

First add the tests in test.py:

import time

...

  # test if text display function contains the data we want
  def test_output_text(self):
    data_filename = self.params['data_filename']
    data_expected = self.params['data']
    data = stock.get_stock_data_from_file(data_filename, data_expected.keys())
    out = stock.view_as_text(data)
    self.assertIn(data['symbol'], out)
    self.assertIn("${:.2f}".format(data['regularMarketPrice']), out)
    self.assertIn("{:2.2f}%".format(data['regularMarketChangePercent']), out)
    self.assertIn(time.strftime("%d/%m/%Y %H:%M:%S %Z", time.localtime(data['regularMarketTime'])), out)

  # test if html display function contains the data we want
  def test_output_html(self):
    data_filename = self.params['data_filename']
    data_expected = self.params['data']
    data = stock.get_stock_data_from_file(data_filename, data_expected.keys())
    out = stock.view_as_html(data)
    self.assertIn(data['symbol'], out)
    self.assertIn("${:.2f}".format(data['regularMarketPrice']), out)
    self.assertIn("{:2.2f}%".format(data['regularMarketChangePercent']), out)
    self.assertIn(time.strftime("%d/%m/%Y %H:%M:%S %Z", time.localtime(data['regularMarketTime'])), out)

The above tests do not account for things like variable precision needed for very small amounts, or handling different currencies and locales, but, this is just a prototype for a specific use case. I can add more use cases later and generalize the code to handle all these stuff, and more.

Then, I add the actual output functions in python.py, to pass the tests:

import time

...

def view_as_text(data):
  template = "{}: ${:.2f} ({}{:2.2f}%) on {}"
  return template.format(data['symbol'],
                         data['regularMarketPrice'],
                         '+' if data['regularMarketChangePercent'] >= 0 else '',
                         data['regularMarketChangePercent'],
                         time.strftime("%d/%m/%Y %H:%M:%S %Z", time.localtime(data['regularMarketTime'])))

def view_as_html(data):
  template = """<!DOCTYPE html>
<html>
  <head>
    <style type="text/css">
      body { font-family:Calibri, Arial, Sans; }
      .price { font-size: normal; }
      .symbol { font-weight:bold; color:blue; }
      .price { font-weight:bold; }
      .timeref { font-size: small; }
    </style>
  </head>
  <body>
    <div id="stock">
      <p class="price"><span class="symbol">{{symbol}}</span>: $<span class="price">{{regularMarketPrice}}</span> <span class="pc">({{regularMarketChangePercent}})</span></p>
      <p class="timeref"><span class="time">{{regularMarketTime}}</span></p>
    </div>
  </body>
</html>"""

  out_html = template
  out_html = out_html.replace('{{symbol}}', data['symbol'])
  out_html = out_html.replace('{{regularMarketPrice}}', "${:.2f}".format(data['regularMarketPrice']))
  out_html = out_html.replace('{{regularMarketChangePercent}}', "{}{:2.2f}%".format('+' if data['regularMarketChangePercent'] >= 0 else '',
                                                                                    data['regularMarketChangePercent']))
  out_html = out_html.replace('{{regularMarketTime}}', time.strftime("%d/%m/%Y %H:%M:%S", time.localtime(data['regularMarketTime'])))
  return out_html

I can already see some optimizations that could be done, but I will leave the refactoring for later.

Run the tests and see them pass:

python test.py
....
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

More prototyping – read from API

Now that I have some working prototype code, let’s add some code to read live data from the API!

Add this test in test.py:

 # test if input from api read contains the expected fields and we were able to read them
  def test_read_from_api(self):
    data_expected = self.params['data']
    data = stock.get_stock_data_from_api(data_expected['symbol'], data_expected.keys())
    for field, value in data_expected.items():
      self.assertIn(field, data)

Run the test, fail, then implement the missing function in stock.py:

import requests

...

# read stock data from Yahoo Finance API,
# find specified keys and return their values in a dictionary
def get_stock_data_from_api(symbol, keys = None):
  if not keys:
    keys = [ 'symbol', 'regularMarketPrice', 'regularMarketChangePercent', 'regularMarketTime' ]
  # prepare url and some headers to keep the api happy
  url = "https://query1.finance.yahoo.com/v7/finance/quote?symbols={}".format(symbol)
  headers = { 'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36' }

  # handle the request to the API
  try:
    response = requests.get(url, headers=headers)
    if response.status_code != 200:
      print("error getting stock price: {} - {}".format(response.status_code, response.reason))
      return None
  except requests.exceptions.RequestException as e:
    print("error getting stock price: {}".format(e))
    return None

  # handle the data returned from the API
  try:
    data_in = json.loads(response.text)
    if data_in['quoteResponse']['error']:
      print("error from api: {}", data_in['quoteResponse']['error'])
      return None
    data_out = {}
    for key in keys:
      data_out[key] = data_in['quoteResponse']['result'][0][key]
    return data_out
  except IndexError as e:
    print("error getting data from API for {}: no data".format(symbol))
    return None
  except KeyError as e:
    print("error getting data from API for {}: {}".format(symbol, repr(e)))
    return None

There’s some quick and dirty error handling code in there, just to cover my bases and know what happened in case something goes wrong.

Also, there’s some repeated code from the file reading function, and I should probably split the function in two (one for reading from the API and one for extracting the data) but I’ll deal with code optimization and cleaning later.

If I run the test, it will fail, because I don’t have the requests module installed, as I started with a new virtual environment:

ModuleNotFoundError: No module named 'requests'

Let’s install it (dependencies will be installed automatically) and update the environment’s requirements:

pip install requests
pip freeze > requirements.txt

Now, let’s run the test again:

python test.py
.....
----------------------------------------------------------------------
Ran 5 tests in 0.689s

OK

Wrap up the prototype

So, with all the tests and prototype code done, I can now make a simple console app in get_stock_price.py:

import sys
import stock

# read command line argument as stock to query for
symbol = sys.argv[1] if len(sys.argv) > 1 else 'MSFT'

if symbol == 'test':
  data = stock.get_stock_data_from_file('test_api_sample_msft.json')
else:
  data = stock.get_stock_data_from_api(symbol)

if data:
  print(stock.view_as_text(data))

Now, I can run this and enjoy the results of my hard work:

python get_stock_price.py test
MSFT: $255.02 (+0.13%) on 02/12/2022 23:00:04 EET
python get_stock_price.py MSFT
MSFT: $245.42 (-0.80%) on 09/12/2022 23:00:04 EET
python get_stock_price.py AAPL
AAPL: $142.16 (-0.34%) on 09/12/2022 23:00:04 EET
python get_stock_price.py TLNL
error getting data from API for TLNL: no data

I will now commit the code as a working local prototype and push to GitHub repository:

git add .
git commit -m 'first working local prototype'
git push

Let’s cloud it

Now that I have the local prototype, I can move on with setting up the Azure Functions environment and code.

This will be the subject of the next part of this series: Lab 001 – Solving a simple use case with Azure Functions – Part 2.

Solving a simple use case with Azure Functions – Part 1

The workflow

The output

Analyze the source API

How to read and parse the data

Create a project to work with

Make a prototype

More prototyping – display data in simple text and HTML

More prototyping – read from API

Wrap up the prototype

Let’s cloud it

Tags:

greg

Leave a Reply Cancel reply

Hello world!

Solving a simple use case with Azure Functions – Part 2