securing API with PEP proxy

Introduction

SDK part 1 of the documentation introduced us to the "How to use APIs for Context Manager (Orion) and Time Series (QuantumLeap)". In those documents we have seen how anybody could query any type of resources in the platform. That is, as you may imagine, not great in terms of security.
For demo'ing purposes, we left port 1026 open, so you could query services directly, now, for the production platform that won't be the case, all your requests will need to be authorized, for understanding how to do this we have wrote SDK part 2.

SDK part 2, introduces us to using those same APIs (and others) "securely". By securely here we mean: the user making the request is authenticated -platform validates who you are-, but also getting authorization for our query -platform validates (or not) the action you want to do-.
The latter is what is called access control, which in simple words means checking whether you are authorized or not to retrieve, delete or modify a resource (*) of the platform.
Access controlled is enforced by PEP Proxy (Wilma), this service runs on port 1027, this is where all your requests are going to be sent to.

(*) resource = FIWARE's entity

MUST READS

Before getting started with this tutorial, we recommend reading FIWARE tutorial on interacting with IdM (Keyrock) and PEP Proxy (Wilma): https://fiware-idm.readthedocs.io/en/latest/oauth/oauth_documentation/index.html

If you want to know about how to secure one of your components with PEP Proxy (Wilma) please refer to: https://fiware-tutorials.readthedocs.io/en/latest/pep-proxy/index.html#logging-in-to-keyrock-using-the-rest-api

Prepare environment

After cloning the git repo please prepare your environment

export KEYROCK_HOST=5.53.108.182
export ORION_HOST=5.53.108.182
export PROXY_HOST=5.53.108.182
cd scripts

Note, here we use different HOST IPs cause each service could run in different machines.
This is also so the reader understands with which service is being used at each time when lunching a script.
For simpler configuration we will have later on a single entrypoint, "the naiades server", which will run every service in the same machine, no need to set up several all this variables.

User credentials

Each platform client uses credentials (email and password) for authenticating against the identity manager and PEP Proxy(IdM and Wilma)

We provide testing credentials, but be aware that those are not going to work in production :)

city-pilot-1@example.com : test
wms-1@example.com : test

So at some point you should ask the IdM maintainer (UDGA or SIMAVI) to create credentials for you.

Priviledges

city-pilot-1@example.com -> can READ and can WRITE into FlowerBed entity
wms-1@example.com -> can READ and cannot WRITE into FlowerBed entity

Oauth2 flow and http `Authorization` header

For this demo, we use grant type password for the Oauth2 app, this means that the app secret is not private, it is shared in clear text publicly in this repo scripts (Authorization http header).
This is not very safe, but for demo purposes it should be fine, we may request you to generate this on your own later on.
If you plan to implement your own app which issues tokens then you should use the standard OAuth2 flows described in the link MUST READ section,

The Authorization header is Authorization: Basic NDU3ODhiM2YtMzRjNy00YThlLTkwZGMtZGZiODdlOGFkMGNjOjVmMmI0YTQ5LTJkMDUtNDQ2Ny04NDQ4LTI1ZDA0OWQwMzQ5OQ==

GET /token using credentials

Please look at the first script created for getting a token with curl:

Request

>> cat security_01_get_token_with_password.sh

curl -iX POST \
  "http://$KEYROCK_HOST:3005/oauth2/token" \
  -H 'Accept: application/json' \
  -H 'Authorization: Basic NDU3ODhiM2YtMzRjNy00YThlLTkwZGMtZGZiODdlOGFkMGNjOjVmMmI0YTQ5LTJkMDUtNDQ2Ny04NDQ4LTI1ZDA0OWQwMzQ5OQ==' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data-raw 'grant_type=password&username=city-pilot-1@example.com&password=test&scope=permanent'

Response

Now, lets try it out using the development deployment of keyrock:

>> ./security_01_get_token_with_password.sh

Querying Identity Managed (Keyrock) at: 5.53.108.182
HTTP/1.1 200 OK
Cache-Control: no-cache, private, no-store, must-revalidate, max-stale=0, post-check=0, pre-check=0
Content-Type: application/json; charset=utf-8
Content-Length: 103
ETag: W/"67-ClUkPCX8d82NxI49F/Bx3jRZwN8"
Set-Cookie: session=eyJyZWRpciI6Ii8ifQ==; path=/; expires=Fri, 12 Jun 2020 14:56:17 GMT; httponly
Date: Fri, 12 Jun 2020 13:56:17 GMT
Connection: keep-alive

{"access_token":"b8fb5c456f4cd518b84788e35bbbd2538bf262be","token_type":"Bearer","scope":["permanent"]}

(!) Please export obtained  <access_token> as KEYROCK_TOKEN, e.g. 'export KEYROCK_TOKEN=<put_the_received_access_token_here!>'

As indicated in the output of this command, you need to export the access_token as KEYROCK_TOKEN, which will enable you to get resources from the IoT Platform.

GET /entity attribute value, without a token (seeing it FAIL)

First, lets see what would happen if we didnt use the token for getting a entity from the context manager:

Request

>> cat security_02_request_without_token.sh
curl -iX POST \
  "http://$KEYROCK_HOST:3005/oauth2/token" \
  -H 'Accept: application/json' \
  -H 'Authorization: Basic NDU3ODhiM2YtMzRjNy00YThlLTkwZGMtZGZiODdlOGFkMGNjOjVmMmI0YTQ5LTJkMDUtNDQ2Ny04NDQ4LTI1ZDA0OWQwMzQ5OQ==' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data-raw 'grant_type=password&username=city-pilot-1@example.com&password=test&scope=permanent'

Response

>> ./security_02_request_without_token.sh
Querying Fiware entrypoint (PEP_PROXY) at: 5.53.108.182
Auth-token not found in request header%

This is normal, this means that the PEP proxy (Wilma) didnt find any authentication information on the request.
In simple words, this is: "you cannot open the door if you dont have the key"

GET /entity attribute value, with a token

Request

curl -X GET \
  "http://$PROXY_HOST:1027/v2/entities/urn:ngsi-ld:FlowerBed:FlowerBed-1/attrs/soilMoistureVwc/value"\
  --header "Fiware-Service: carouge" \
  --header "Fiware-ServicePath: /Watering" \
  --header "X-Auth-Token: $KEYROCK_TOKEN"

Response

>> ./security_03_request_with_token.sh
Querying Fiware entrypoint (PEP_PROXY) at: 5.53.108.182
8%

Great! soil moisture value is 8, let's try to write into that value

PUT /entity attribute value, with a token

Lets update moisture value to a random number:

Request

curl --location --request PUT \
    "http://$PROXY_HOST:1027/v2/entities/urn:ngsi-ld:FlowerBed:FlowerBed-1/attrs/soilMoistureVwc/value"\
    --header "Fiware-Service: carouge" \
    --header "Fiware-ServicePath: /Watering" \
    --header "X-Auth-Token: $KEYROCK_TOKEN" \
    --header "Content-Type: text/plain" \
    -d $(( RANDOM % 10 ))

Response


>> ./security_04_update_entity_with_token.sh
Querying Fiware entrypoint (PEP_PROXY) at: 5.53.108.182


If you didnt get any error message, then you can re-run script <security_04...>.sh you will get a different value for the entity

PUT request was accepted, now lets see what inside of that value again:

>>./security_03_request_with_token.sh

Querying Fiware entrypoint (PEP_PROXY) at: 5.53.108.182
2%

Woo-hoo! we managed to update an entity attribute with our token! We have been granted to do this cause our token was generate with a user with privileges to do so (access control).

How is this handled in the backend? What's under the hood??

Here there is a snapshot of the keyrock dashboard for this (for those of you who are curious about how this is managed) :

configuring roles:

configuring permissions:

Advanced: users without writing rights to write certain entities (wms)

Lets do the same tests we did before, but now using a token with less priviledges that before:

Steps:

change in script security_01... user to wms-1@example.com, same password.
repeat security_01..., update env var for the token
run security_03... returns requested value, great ..
lets try to write that value with ./security_04_update_entity_with_token.sh
it returns: User access-token not authorized

Viola! our wms-1 user doesnt have the right to write to this specific entity, this demonstrates how access control works in the NAIADES context.

Advanced: using the JWT scope

Instead of using a bearer token, we can use JWT, which has some added advantages, this demonstrates one of them:

Request token using jwt

Note we can use &scope=jwt as query param:

curl -iX POST \
  "http://$KEYROCK_HOST:3005/oauth2/token" \
  -H 'Accept: application/json' \
  -H 'Authorization: Basic NDU3ODhiM2YtMzRjNy00YThlLTkwZGMtZGZiODdlOGFkMGNjOjVmMmI0YTQ5LTJkMDUtNDQ2Ny04NDQ4LTI1ZDA0OWQwMzQ5OQ==' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data-raw 'grant_type=password&username=city-pilot-1@example.com&password=test&scope=jwt'

Response jwt

Returns a bigger token containing our roles in the platform!

>> ./security_01_get_token_with_password_jwt.sh
Querying Identity Managed (Keyrock) at: 5.53.108.182
HTTP/1.1 200 OK
Cache-Control: no-cache, private, no-store, must-revalidate, max-stale=0, post-check=0, pre-check=0
Content-Type: application/json; charset=utf-8
Content-Length: 1013
ETag: W/"3f5-jf2XD7ISOVcd9Zb0Y2hn/CYVWoY"
Set-Cookie: session=eyJyZWRpciI6Ii8ifQ==; path=/; expires=Tue, 16 Jun 2020 15:31:51 GMT; httponly
Set-Cookie: session.sig=TqcHvLKCvDVxuMk5xVfrKEP-GSQ; path=/; expires=Tue, 16 Jun 2020 15:31:51 GMT; httponly
Date: Tue, 16 Jun 2020 14:31:51 GMT
Connection: keep-alive

{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJvcmdhbml6YXRpb25zIjpbeyJpZCI6ImJjNmIzNjIxLWM0NGYtNGI1YS1hZmYwLWYwYTVjMGFiZmMyZCIsIm5hbWUiOiJuYWlhZGVzLWRhdGEtY29sbGVjdG9ycyIsImRlc2NyaXB0aW9uIjoibmFpYWRlcyBJb1QgZGF0YSBjb2xsZWN0b3IgY29tcG9uZW50cyIsIndlYnNpdGUiOm51bGwsInJvbGVzIjpbeyJpZCI6IjkyM2U5ZTcxLTBjYjAtNDBjYi05ZDZhLTk2ZWUxNmVjMjBkOSIsIm5hbWUiOiJkYXRhLWNvbGxlY3RvciJ9XX1dLCJkaXNwbGF5TmFtZSI6IiIsInJvbGVzIjpbXSwiYXBwX2lkIjoiNDU3ODhiM2YtMzRjNy00YThlLTkwZGMtZGZiODdlOGFkMGNjIiwidHJ1c3RlZF9hcHBzIjpbXSwiaXNHcmF2YXRhckVuYWJsZWQiOmZhbHNlLCJpbWFnZSI6IiIsImVtYWlsIjoiY2l0eS1waWxvdC0xQGV4YW1wbGUuY29tIiwiaWQiOiJlYWU1MmZiZS1kYzQ2LTQ2ODUtOWQzYi1kZTE3Zjk5N2UwOTEiLCJhdXRob3JpemF0aW9uX2RlY2lzaW9uIjoiIiwiYXBwX2F6Zl9kb21haW4iOiIiLCJlaWRhc19wcm9maWxlIjp7fSwiYXR0cmlidXRlcyI6e30sInVzZXJuYW1lIjoiY2l0eS1waWxvdC0xIiwidHlwZSI6InVzZXIiLCJpYXQiOjE1OTIzMTc5MTEsImV4cCI6MTU5MjMyMTUxMX0.nqurtvdVMaJ_oqnyNkBk_c-m7MuivZ2uZCouuh1eyX0","token_type":"jwt","refresh_token":"899663e3c36f9b2a9d8f3582f4d12d0d30246401","scope":["jwt"]}

(!) Please export obtained  <access_token> as KEYROCK_TOKEN, e.g. 'export KEYROCK_TOKEN=<put_the_received_access_token_here!>'

decoding jwt with jwt.io

If you check on the right column we have information on what are our roles in the NAIADES platform

Contact

If you have spotted any errors on the doc, or something doesnt work as expected please contact fsismondi@udgalliance.org

Introduction

MUST READS

Prepare environment

User credentials

Priviledges

Oauth2 flow and http Authorization header

GET /token using credentials

Request

Response

GET /entity attribute value, without a token (seeing it FAIL)

Request

Response

GET /entity attribute value, with a token

Request

Response

PUT /entity attribute value, with a token

Request

Response

How is this handled in the backend? What's under the hood??

Advanced: users without writing rights to write certain entities (wms)

Steps:

Advanced: using the JWT scope

Request token using jwt

Response jwt

decoding jwt with jwt.io

Contact

Oauth2 flow and http `Authorization` header