MT4007 HT23

Appearance

Lecture 9 (REST APIs) ​

Resources ​

You can find examples and motivation in the resources.

Summary ​

In this lecture on REST APIs, we will be diving into the core principles REST as a communication framework for applications and we will learn how to practically work with REST APIs. The goal is to understand the standardized and efficient method for retrieving resources through network communication.

REST APIs ​

REST (Representational State Transfer Application) APIs is a communications framework for applications, typically, web applications. In REST the application (requesting) resources is called the client and the application that responds is called the server. For instance, when you are at a restaurant, you are requesting food, which makes you the client. The restaurant serves you the food that you have requested. That is, the restaurant responds to you request. This makes the restaurant the server.

The REST framework specifies rules on how consitutents communicate in a network. There are several other frameworks for communication but REST is standard.

The Framework ​

In REST, everything is a resource. Resources are identified by unique URIs (Uniform Resource Identifiers). The clients in the network communicate using HTTP methods. E.g GET to retrieve resources, POST to create new resources, PUT to update resources, DELETE to remove resources. There are several more HTTP methods but these are the most common ones. Resources have one or more representations, which can be in various formats such as JSON or XML.

Each request from a client to a server must contain all the information need to understand and fulfill the request. The server doesn't store any infomation about the client inbetween requests. This makes REST stateless communications framework.

The idea is that everytime you make a request, you get a response containing some resource(s). For instance, imagine we are searching for something on google, we are in other words making a request to googles servers, and the server responds with the search results. The REST pattern is everywhere on the internet. The figure below illustrates this idea.

rest-api

Image from: https://www.altexsoft.com/blog/rest-api-design/

Before we continue on about REST APIs, we should consider why this is important to learn. For data scientist, most of the data will be provided from an organizations through the internet. It is highly unlikely, that you will be handed a usb containing data files. Therefore, retrieving data in a standardised format is essential. REST is the standard way of retrieving resources through a network.

Another motivation to why we should learn about REST APIs is because of the scope of the tool. REST is an efficent way to communicate in a network and the type of communication and network is arbitrary. In the motivation above, the resource was data but it can be anything. For instance, smart lights communicate using REST. We make a request from our mobile app to turn on/off a specific light. The app sends a request to the lamp server and the resource is then a physical action. This is how IKEA smart home works. This is just scratching the surface of the possibilities for REST APIs.

Request ​

The request has the following structure that needs to be specified. The request consists of a

  • URI: Specifying the location of the resource you are requesting.
  • Method: Specifying the type of request. E.g. GET, POST, DELETE, PUT and so on.
  • Header: The headers provide additional information about the request or the client. The information needed is dependent on the API.
  • Body: Optionally, data can be sent with the request, this is passed in the body of the request.

Response ​

The response consists of a

  • Status: Information of the status of the request. For instance status code
  • Header: The header provide additional information about the response or the server. The information needed is dependent on the API.
  • Body: The response consists of a body that contains the resource(s) requested (if request has been accepted).

Resources ​

The resource is what we are after, but they can come in different types of representations.

Representation ​

The resource can be arbitrary but it can be of the form of some data in this case it is usually represented in a standard format. Here are some common formats.

JSON ​

JavaScript Object Notation (JSON) is a lightweight data interchange format that is easy for humans to read and write, and easy for machines to parse and generate. JSON represents data as key-value pairs, similar to how objects are represented in python. Each key is a string, and values can be strings, numbers, objects, arrays, booleans, or null. See the following JSON representation of an individual.

json
{
  "name": "John Doe",
  "age": 30,
  "isStudent": false,
  "address": {
    "city": "Exampleville",
    "country": "JSONland"
  },
  "hobbies": ["reading", "coding", "traveling"]
}
{
  "name": "John Doe",
  "age": 30,
  "isStudent": false,
  "address": {
    "city": "Exampleville",
    "country": "JSONland"
  },
  "hobbies": ["reading", "coding", "traveling"]
}

Again, it is very easy to read and parse. This is a very typical representation of a resource.

XML ​

eXtensible Markup Language (XML), is a markup language designed to store and transport data. It is a flexible format that is both human-readable and machine-readable. See the following XML representation of an individual.

xml
<person>
  <name>John Doe</name>
  <age>30</age>
  <isStudent>false</isStudent>
  <address>
    <city>Exampleville</city>
    <country>XMLland</country>
  </address>
  <hobbies>
    <hobby>reading</hobby>
    <hobby>coding</hobby>
    <hobby>traveling</hobby>
  </hobbies>
</person>
<person>
  <name>John Doe</name>
  <age>30</age>
  <isStudent>false</isStudent>
  <address>
    <city>Exampleville</city>
    <country>XMLland</country>
  </address>
  <hobbies>
    <hobby>reading</hobby>
    <hobby>coding</hobby>
    <hobby>traveling</hobby>
  </hobbies>
</person>

Again XML is easy to read and parse. Less common than JSON but still very common.

Example ​

The internet communicates using the REST framework. In this case the request is a website and the response html. The browser then parses the html and shows you the website. Under the hood, you web browser (e.g Chrome) makes a GET request to some server on the network(internet) and the server responds with the code for the website.

Let's try to make our own request without using the browser. We can get the course schedule information from the Time Edit page.

The URI: https://cloud.timeedit.net/su/web/stud1/ri167XQQ508Z50Qv57093gZ6y3Y7806Q5Y65Y1.json. This is where the resource is located. Furthermore, we need to specify the method type. In this case we are making a GET request.

The request

Python
import requests 

# Making a get requests to URI
URI = "https://cloud.timeedit.net/su/web/stud1/ri167XQQ508Z50Qv57093gZ6y3Y7806Q5Y65Y1.json"
response = requests.get(URI)

data = response.json()
import requests 

# Making a get requests to URI
URI = "https://cloud.timeedit.net/su/web/stud1/ri167XQQ508Z50Qv57093gZ6y3Y7806Q5Y65Y1.json"
response = requests.get(URI)

data = response.json()
R
# Since R does not naturally have a similar structure to json, we need a few
more libaries to work with JSON.

library(httr2)
library(jsonlite) # Parse JSON
library(listviewer) # View Json

URI <- "https://cloud.timeedit.net/su/web/stud1/ri167XQQ508Z50Qv57093gZ6y3Y7806Q5Y65Y1.json"
request <- request(URI)
response <- req_perform(request)
json <- response %>% resp_body_string
jsonedit(json) # View the JSON in  a nice way

data <- fromJSON(json)
# Since R does not naturally have a similar structure to json, we need a few
more libaries to work with JSON.

library(httr2)
library(jsonlite) # Parse JSON
library(listviewer) # View Json

URI <- "https://cloud.timeedit.net/su/web/stud1/ri167XQQ508Z50Qv57093gZ6y3Y7806Q5Y65Y1.json"
request <- request(URI)
response <- req_perform(request)
json <- response %>% resp_body_string
jsonedit(json) # View the JSON in  a nice way

data <- fromJSON(json)

The response body contains the following data:

json
{'columnheaders': ['Lokal',
  'Karta',
  'Kurs',
  'Kurstillfälle',
  'Delkurs, Moment',
  'Aktivitet',
  'Studentgrupp',
  'Lärare',
  'Information',
  'Litteraturinfo',
  'Tentamenssal',
  'URL',
  'Viktigt meddelande'],
 'info': {'reservationlimit': 1000, 'reservationcount': 7},
 'reservations': [{'id': '2612025',
   'startdate': '2023-12-01',
   'starttime': '13:00',
   'enddate': '2023-12-01',
   'endtime': '17:00',
   'columns': ['Lärosal 17. Albano Hus 2. Vån 2',
    'https://www.openstreetmap.org/?mlat=59.35523&mlon=18.05639#map=18/59.35523/18.05639',
    'MT4007',
    '48182. H23. 50%. DAG. NML',
    '',
    'Föreläsning',
    '',
    'Taariq Fahran Nazar',
...
{'columnheaders': ['Lokal',
  'Karta',
  'Kurs',
  'Kurstillfälle',
  'Delkurs, Moment',
  'Aktivitet',
  'Studentgrupp',
  'Lärare',
  'Information',
  'Litteraturinfo',
  'Tentamenssal',
  'URL',
  'Viktigt meddelande'],
 'info': {'reservationlimit': 1000, 'reservationcount': 7},
 'reservations': [{'id': '2612025',
   'startdate': '2023-12-01',
   'starttime': '13:00',
   'enddate': '2023-12-01',
   'endtime': '17:00',
   'columns': ['Lärosal 17. Albano Hus 2. Vån 2',
    'https://www.openstreetmap.org/?mlat=59.35523&mlon=18.05639#map=18/59.35523/18.05639',
    'MT4007',
    '48182. H23. 50%. DAG. NML',
    '',
    'Föreläsning',
    '',
    'Taariq Fahran Nazar',
...

It is a simple as that! To get the data into a nice format we need to wrangle the raw data. This however we already know how to do.