GSoC 2022 [Matrix Cerberus]

#10 Tenth Week [Are you my patron?]

noreply@blogger.com (Munukurthi Binesh) — Wed, 24 Aug 2022 10:57:00 +0000

This week I've finished integrating the second third party service for the app - Patreon. Now, a creator can choose which tier members for their campaign have access to a particular room.

TL;DR

Design the UI for modal pop ups to save and display room conditions for patreon.
Connect the backend with the frontend UI

Related PRs

#54 - Design Patreon UI and connect with backend

Demo Pages

#9 Ninth Week [Bugs can be scary!]

noreply@blogger.com (Munukurthi Binesh) — Tue, 23 Aug 2022 07:33:00 +0000

I've encountered a couple of unexpected bugs this week with the GitHub API and spent majority of the time trying to fix them.

TL;DR

Fix bugs related to insufficient permissions to fetch data from GitHub API.
Rewrite background validator task to run less frequently and after long intervals.
Support for queues to trigger a background validation check everytime a room condition is edited or a new user is added.

Related PRs

#49 - Patreon OAuth
#56 - Trigger validation checks when a new condition is added

Long version

I had a call with my mentors this week to discuss about the project's overall progress and I got the chance to show a working example for the project.

While I was presenting I noticed a couple of unexpected bugs with the background validator.

The "Bug"

In my previous posts, I talked about how the background validator works in general but I didn't go in detail on how the data from third party accounts is fetched and how they are compared with the conditions saved in the room account data.

Let's take an example and go over it now.

Consider that we are dealing with the repository conditions for a GitHub repository under an owner (user/org) saved in the room account data and would like to validate a particular user's room membership.

We start by fetching the GitHub access token for the current user being validated and try to fetch data related to repository permissions. But the problem comes when the user doesn't have push access for the repository. In that case, we end up with the error "Must have push access to view collaborator permission."

So, what do we do?
We will have to handle it differently based on the owner. If the owner is a user, we can just use the ir access token and get the relevant permissions.

You might be wondering "how can we say for sure that we'll have the access token of this user?". Well, only the owner has the ability to add repository conditions that they own for a room and if a repository condition is saved under a room then obviously the user must've registered with our application.

What if the owner is an organisation instead of a user?

This is where it gets tricky. We will have to keep track of the last user who has added/edited the room condition by saving their GitHub username along with the room conditions data.

This way we can always ensure that we never run into "Must have push access to view collaborator permission" problem.

How do I make it less frequent?

Until now, the background validator gets triggered every 10 seconds. This is too frequent, unnecessary and can be very resource consuming.

So, my mentor suggested to run it every 1 hour or so and make it less frequent.

But a user/room admin will have to wait for an hour until the updated changes can take effect every time they edit room conditions.

To handle this problem, we made use of asyncio-queue to process the recently edited room conditions in a FIFO manner.

#8 Eighth Week [One down, One more to go!]

noreply@blogger.com (Munukurthi Binesh) — Sun, 14 Aug 2022 19:38:00 +0000

This week after adding some finishing touches, I've successfully managed to add support for the GitHub! Now, it is time to implement the second third party service - "Patreon".

TL;DR

Add toast notifications wherever that is necessary.
Patreon OAuth 2.0

Related PRs

#48 - Connect backend and frontend for Github Conditions Page
#49 - Patreon OAuth

#7 Week 6-7 [Life is unpredictable]

noreply@blogger.com (Munukurthi Binesh) — Sun, 14 Aug 2022 19:28:00 +0000

Due to some health related issues the progress of this project has greatly slowed down during these weeks.
So, I'll be including both the week's progress in this blog post.

TL;DR

Integration tests for Login page
UI to select options in Room and Github Conditions pages
Fetch and save/update data related to room conditions from the client side

Related PRs

#6 Fifth Week (My Room, My Rules)

noreply@blogger.com (Munukurthi Binesh) — Sat, 23 Jul 2022 06:06:00 +0000

During the first half of the week, I worked on designing the UI for the room conditions and the external url invites page. And on the latter part of the week, I focused on writing tests for the backend and the frontend.

TL;DR

Room Conditions UI
External URL page

Related PRs

#39 - Add Room and Room External URL webpages

Demo Pages

Room Settings UI

Select Third Party Account

External URLs page

#5 Fourth Week (Show me your pass, please?)

noreply@blogger.com (Munukurthi Binesh) — Mon, 11 Jul 2022 19:05:00 +0000

I've spent most of my time implementing the background task which validates a user's room membership to a particular room.

TL;DR

Background validator task
Global app state for the backend

Related PRs:

Long Version:

How does the background task validate a user's room membership?

The background task verifies all the users currently registered with the app and it runs at uniform intervals (a specific time period
It starts by fetching the users and rooms list from `matrix-cerberus.global_data` event type in account data.
Then the bot has two loops iterating over the users and the rooms to gets a user's membership in a particular room.

For join and invite states, I've tried to use the data which is cached internally by matrix-nio on every successful sync with the matrix homeserver.
When it comes to the other states such as ban, knock and leave the bot will try to fetch the m.room.member room state event with the user_id as state_key.
But, it feels like there's something missing in here. Don't you think?
Well, you are right.
We are not keeping track of users who have either:

Rejected an invite
Retracted their knock
User left the room
Was kicked by a member who is not a bot.

In all these cases, we are not supposed to send an invite to the user even if they satisfy the room conditions.

Why?
- In the first two cases, we want to avoid spamming the users with invites
- Whereas, in the third and fourth case, we don't want to go against a room admin's/user's wishes and invite the user to the room.
There is actually a simple solution which will handle all the above cases. We can just fetch the full state of the m.room.member state event for all the users who have left the room via the /rooms/{roomId}/members endpoint.
With this, we get the sender property. And if the sender is not equal to the bot user_id we can simply ignore the user.

How does this solve the problem?

In the first three cases, the sender key is equal to the state key i.e. user_id.
In the fourth case, the sender key can be anyone other than the bot.
So, if we can ignore users whose sender key in m.room.member state event is not equal to the bot's user_id we can avoid all those 4 scenarios.

4. Now, that we have the user's current room membership the bot will use the methods from github api class and check for the conditions set in the matrix- cerberus.rooms.<room_id>.

       5. If any of the conditions are satisfied the bot will send corresponding invite and                    kick event for the room.
       6. Steps 4 and 5 are repeated for each user_id and room_id combination.
       7. After the entire loop finishes executing, the bot puts this task to sleep for a specific            amount of time and then fetches the new updated rooms and users list before                    starting the all over again from 1.

A global app state for backend?

After reviewing PR Feat: external url invites, Cadair has suggested to use a global app state variable which takes care of maintaining all the global variables like http_client, session_storage, bot_client and so on where it is necessary to maintain a single session for each Class. It is an alternative to using the global variables independently which introduces bugs and makes the code unmaintainable when the amount of code scales up.

So, I ended up refactoring the entire code to support the global_app_state variable such that the session is created during the app startup and stop the session on app shutdown.

I've tried to make sure the core modules are not dependent on other module's global variables thus making them completely independent and are at the top of the tree during imports.

#4 Third week (GitHub, GitHub, GitHub...)

noreply@blogger.com (Munukurthi Binesh) — Mon, 04 Jul 2022 19:23:00 +0000

This week I mostly focused on how the bot will be interacting with the GitHub API from a user's POV.

TL;DR

Setup OAuth for GitHub
GitHub API class to interact with the GitHub REST and GitHub GraphQL APIs.
Methods to manage user and global account data events on a matrix server.

Related PRs:

#3 Second week (External URL Invites)

noreply@blogger.com (Munukurthi Binesh) — Sun, 26 Jun 2022 18:58:00 +0000

Hey there!

This is my second week's blog post during Google Summer of Code 2022.

TL;DR

Add a background task that runs the bot in parallel to the web server which handles API requests.
Support for Aiohttp session.
Implement External Url Invites
Add dependency to handle unauthorized users.
Create custom router and axios interceptor to parse responses and redirect users to login page on 401 unauthorized request error.

Related PRs:

#2 First Week (CODING PERIOD STARTS :D)

noreply@blogger.com (Munukurthi Binesh) — Mon, 20 Jun 2022 19:05:00 +0000

Hey there!

This is my first week's blog post during Google Summer of Code 2022.

TL;DR

Design UI for Login page
Add auto discovery of homeservers feature for users
Implement password and sso based login.
Authenticate with OpenID tokens.
Setup pre-commit and CI workflow for testing and linting.

Long version

I tried to get a head start during the community bonding period by working on the Login Page UI. And fortunately, I was able to finish it just before the first coding period started :D

Most of my inspirations for the Login page came from the Element web Login page.

Desktop view

Mobile view

How does the login work?

Ok, now enough about the UI designs and let's look at how the login functionality works.

Usually in a typical web application, a user enters a username and a password. These details will then be passed onto the backend server where it'll be stored in a database.

But in our case, we are not actually registering the user or saving their credentials.

So how are we going to authorize a user?

This is where the OpenID token comes in. According to the OpenID Connect spec, it is used to verify the identity of an end-user on a server without actually giving away any sensitive information during the process.

The steps listed down below give a brief overview on how a user is authenticated with the backend server:

A user fills the form with their username (or any other valid user field) and their password on the client-side (React)
React then sends a request to a matrix homeserver where the current user's account is registered to fetch the access token.
After the access token is received, the frontend then tried to get an OpenID token with the help of the access token.
Now, this OpenID token has only one purpose and that is to verify the authenticity of a user. This OpenID token is then sent to the backend via the /api/verify-openid endpoint.
The backend then tries to verify if the current user has a matrix account registered in a homeserver by sending the open id token to this endpoint GET /_matrix/federation/v1/openid/userinfo
After the backend receives a success response along with the current user's matrix ID, it creates a new server session for the user and returns the session id back to the frontend which will be used to maintain connection until the session expires.

#1 Communiy Bonding Period

noreply@blogger.com (Munukurthi Binesh) — Fri, 03 Jun 2022 17:04:00 +0000

Hola Amigo!

Welcome to my first blog post during the Google Summer of Code 2022.

Summary

I've managed to do a lot during the Community bonding period :). I had several discussions with my mentor on choosing the right framework and tools for the project.

In the end, we finalized on the following setup

React + Tailwind CSS for frontend
FastAPI for backend
Nginx which acts as a reverse proxy and serves frontend build.
Redis database to store server sessions.

The final product will be deployed with the help of docker.

Why we chose FastAPI and not Django?

Initially, I planned to use Django framework as backend as presented in my project proposal. Django is mostly known for its "batteries-included" nature where it takes care of most of the security features like user authentication and CSRF tokens for forms. It also has a very good ORM support and not to forget it is a battle-tested framework.
But like every other framework it has its cons - It is very opinionated framework and dosen't have good async support.

And when it comes to FastAPI, it has a very good async support and is very flexible (too flexible in fact). It has its own cons too - relatively new framework and requires the developer to configure most of the important functionalities like server-sessions and user authentication.

In the end, we chose FastAPI since we wont be needing ORM and default Admin page which Django provides as most of our data will be stored in the matrix servers as account data.

#Week 1 of Community Bonding Period:

Since, this is a blank slate project the most important thing is to decide on the name of the project and the logo.

I came up with a list of names (some were crazy though XD) and me and my mentors decided to name our project matrix-cerberus. (Cerberus the greek hellhound which guards the gates of HELL!)

After that I finished creating the boilerplate code for the project and hosted it on https://cerberus.cadair.dev/ (do check it out :D)

#Week 2 of Community Bonding Period:

I use express-sessions a lot in my Node JS projects and I tried to find an exact implementation in FastAPI. Unfortunately, I couldn't find a proper one. So, I decided to make one myself (inserts thanos meme).

I spent major portion of this week trying to configure server-sessions by saving the current user session in a redis database. You can find the implementation here.

#Week 3 of Community Bonding Period:

During the last week, I was mostly trying to refine my project proposal and prepare a new action plan (timeline) for the upcoming Coding Period.

I am also pretty new to some of the tech stack (Docker, FastAPI). So, I mostly used this period to go over the Documentation and some reference implementations.

Introduction

noreply@blogger.com (Munukurthi Binesh) — Fri, 15 Apr 2022 11:58:00 +0000

Hey there! I'm Binesh and I've been selected to work under Matrix.org as a GSoC contributor for this year's summer 2022.
I'll be working on the project "3rd party authorized room membership". It delegates a user's membership for a room based on their past interactions with other third party services.

I plan to add support for the following two services:

GitHub
Patreon

My mentors for this project are @Cadair and @Half-Shot.

I'll be posting about the updates and progress related to the project every week in this blog. I will try to cover the technical details, the challenges I might face during this period and all of my Open Source contributions to Matrix in general.