thesis/content/architecture.tex

\chapter{Framework Architecture}
\section{Core Technology}

It is important to understand that the Tutorial Framework is currently implemented as
two Docker images:
\begin{itemize}
    \item the \texttt{solvable} image is responsible for running the framework and the client
          code depending on it
    \item the \texttt{controller} image is responsible for solution checking (to figure out
          whether the user completed the tutorial or not)
\end{itemize}
During most of this capter I am going to be discussing the \texttt{solvable} Docker image,
with the exception of section \ref{solutioncheck}, where I will dive into how the
\texttt{controller} image is implemented.

The most important feature of the framework is it's messaging system.
Basically what we need is a system where processes running inside a Docker container
would be allowed to communicate with eachother. 
This is easy with lots of possible solutions (named pipes, sockets or shared memory to name a few).
The hard part is that frontend components running inside a web browser --- which could be
potentially on the other side of the planet --- would also need to partake in said communication.
So what we need to create is something of a hybrid between an IPC system and something
that can communicate with JavaScript running in a browser connected to it.
The solution the framework uses is a proxy server, which connects to frontend components
on one side and handles interprocess communication on the other side. 
This way the server is capable of proxying messages between the two sides, enabling
communitaion between them.
Notice that this way what we have is essentially an IPC system in which a web application
can ``act like'' it was running on the backend in a sense: it is easily able to
communicate with processes on the backend, while in reality the web application
runs in the browser of the user, on a completely different machine.

\begin{note}
The core idea and initial implementation of this server comes from Bálint Bokros,
which was later redesigned and fully rewritten by me to allow for greater flexibility
(such as connecting to more than a single browser at a time, different messaging modes,
message authentication, restoration of frontend state, a complete overhaul of the
state tracking system and the possibility for solution checking among other things).
If you are explicitly interested in the differences between the original POC implementation
(which is out of scope for this thesis due to lenght constraints) and the current
framework please consult Bálint's excellent paper and Bachelor's Thesis on it\cite{BokaThesis}.
\end{note}

Now let us take a closer look:

\subsection{Connecting to the Frontend}

The old way of creating dynamic webpages was AJAX polling, which is basically sending
HTTP requests to a server at regular intervals from JavaScript to update the contents
of your website (and as such requiring to go over the whole TCP handshake and the
HTTP request-response on each update).
This has been superseded by WebSockets around 2011, which provide a full-duplex
communication channel over TCP between your browser and the server.
This is done by initiation a protocol handshake using the \texttt{Connection: Upgrade}
HTTP header, which establishes a premanent socket connection between the browser
and the server.
This allows for communication with lower overhead and latency facilitating efficient
real-time applications.

The Tutorial Framework uses WebSockets to connect to it's web frontend.
The framework proxy server is capable to connecting to an arbirary number of websockets,
which allows opening different components in separate browser windows and tabs, or even
in different browsers at once (such as opening a terminal in Chrome and an IDE in Firefox).

\subsection{Interprocess Communication}

To handle communication with processes running inside the container TFW utilizes
the asynchronous distributed messaging library ZeroMQ%
\footnote{\href{http://zeromq.org}{http://zeromq.org}} or ZMQ as short.
The rationale behind this is that unlike other messaging systems such as
RabbitMQ%
\footnote{\href{https://www.rabbitmq.com}{https://www.rabbitmq.com}} or Redis%
\footnote{\href{https://redis.io}{https://redis.io}},
ZMQ does not require a daemon (message broker process) and as such
has a much lower memory footprint while still providing various messaging
patterns and bindings for almost any widely used programming language.
An other --- yet untilized --- capability of this solution is that since ZMQ is capable
of using simple TCP sockets, we could even communicate with processes running on remote
hosts using the framework.

There are various lower level and higher level alternatives for IPC other than
ZMQ which were also considered during the desing process of the framework at some point.
A few examples of top contenders and reasons for not using them in the end:
\begin{itemize}
    \item The handling of raw TCP sockets would involve lot's of boilerplate logic that
    already have quality implementations in messaging libraries: i.e.\ making sure that
    all bytes are sent or received both require checking the return values of the
    libc \texttt{send()} and \texttt{recv()} system calls, while ZMQ takes care of this
    extra logic involved and even provides higher level messaging patterns such as
    subscribe-publish, which would need to be implemented on top of raw sockets again.
    \item Using something like gRPC\footnote{\href{https://grpc.io}{https://grpc.io}}
    or plain HTTP (both of which
    are considered to be higher level than ZMQ sockets) would require 
    all processes partaking in the communication to be HTTP servers themselves,
    which would make the framework
    less lightweight and flexible: socket communication with or without ZMQ does not
    force you to write synchronous or asynchronous code, whereas common HTTP servers
    are either async or pre-fork in nature, which extort certain design choices on code
    built on them.
\end{itemize}

\section{High Level Overview}

Now being familiar with the technological basis of the framework we can now
discuss it in more detail.

\pic{figures/tfw_architecture.png}{An overwiew of the Tutorial Framework}

Architecturally TFW consists of four main components:
\begin{itemize}
    \item \textbf{Event handlers}: processes running in a Docker container
    \item \textbf{Frontend}: web application running in the browser of the user
    \item \textbf{TFW (proxy) server}: responsible for message routing/proxying
          between the frontend and event handlers
    \item \textbf{TFW FSM}: a finite state machine responsible for tracking user progress,
          that is implemented as an event handler called \texttt{FSMManagingEventHandler}
\end{itemize}
Note that it is important to keep in mind that as I've mentioned previously,
the TFW Server and event handlers reside in the \texttt{solvable} Docker container.
They all run in separate processes and only communicate using ZeroMQ sockets.

In the following sections I am going to explain each of the main components in
greater detail, as well as how they interact with each other,
their respective responsibilities,
some of the design choices behind them and more.

\subsection{Frontend}

This is a web application that runs in the browser of the user and uses
multiple WebSocket connections to connect to the TFW server.
Due to rapidly increasing complexity the original implementation (written in
plain JavaScript with jQuery%
\footnote{\href{https://jquery.com}{https://jquery.com}} and Bootstrap%
\footnote{\href{https://getbootstrap.com}{https://getbootstrap.com}}) was becoming
unmaintainable and the usage of some frontend framework became justified.

Several choices were considered, with the main contenders being:
\begin{itemize}
    \item Angular\footnote{\href{https://angular.io}{https://angular.io}}
    \item React\footnote{\href{https://reactjs.org}{https://reactjs.org}}
    \item Vue.js\footnote{\href{https://vuejs.org}{https://vuejs.org}}
\end{itemize}
After comparing the above frameworks we've decided to work with Angular for
several reasons.
One being that Angular is essentially a complete platform that is very well
suitable for building complex architecture into a single page application.
Other reasons included that the frontend of the Avatao platform is also written
in Angular (bonus points for experienced team members in the company).
An other good thing going for it is that Angular forces you to use TypeScript%
\footnote{\href{https://www.typescriptlang.org}{https://www.typescriptlang.org}}
which tries to remedy the issues\cite{JavaScript}
with JavaScript by being a language that transpiles to JavaScript while
strongly encouraging things like static typing or Object Oriented Principles.

\subsection{Messaging}
\subsection{TFW Finite State Machine}
\subsection{Solution Checking}\label{solutioncheck}