Continue writing about architecture

content/architecture.tex

@@ -124,6 +124,92 @@ greater detail, as well as how they interact with each other,
 their respective responsibilities,
 some of the design choices behind them and more.
 
+\subsection{TFW Message Format}
+
+All components in the Tutorial Framework use JSON%
+\footnote{JavaScript Object Notation: \href{https://www.json.org}{https://www.json.org}}
+messages to communicate with eachother.
+These messages must also comply some simple rules specific to the framework.
+Let's inspect further what a valid TFW message might look like:
+
+\begin{lstlisting}[captionpos=b,caption={The TFW JSON message format}]
+{
+    "key": ...an identifier used for addressing...,
+    "data":
+    {
+        ...
+        optional JSON object carrying arbitrary data
+        ...
+    },
+    "trigger": ...optional state change action...,
+    "signature": ...optional HMAC signature for authenticated messages...,
+    "seq": ...sequence number that is automatically inserted by the TFW server...
+}
+\end{lstlisting}
+
+All valid messages \emph{must} include a \texttt{key} field as this is used by the
+framework for addressing: event handlers and frontend components subscribe to one
+or more \texttt{key}s and only receive messages with \texttt{key}s they have
+subscribed to.
+It is possible to send a message with an empty key, however these messages will not
+be forwarded by the TFW server (but will reach it, so in case the target of a message
+is the TFW server exclusively, an empty \texttt{key} can may used).
+
+The rest of the fields are optional, but most messages will carry something
+in their \texttt{data} field.
+The purpose \texttt{trigger} and \texttt{signature} fields will be detailed
+at a later point in this paper.
+
+\subsection{Networking Details}
+
+The default behaviour of the TFW server is that it forwards all messages from coming from
+the frontend to the event handlers and vice versa.
+So messages coming from the WebSockets of the frontend are forwarded to event handlers
+via ZMQ and messages received through ZMQ from event handlers are forwarded to
+the frontend via WebSockets.
+
+The TFW server is also capable of ``reflecting'' messages back to the side they were
+received on (to faciliate event handler to event handler for instance), or broadcast
+messages to all components.
+This is possible by embedding a whole TFW message in the \texttt{data} field of
+an outer wrapper message with a special \texttt{key} that signals to the TFW server that
+this message requires special attention.
+An example of this would be:
+\begin{lstlisting}[captionpos=b,caption={Broadcasting and mirroring TFW messages}]
+}
+    "key": "broadcast",   // or "mirror"
+    "data":
+    {
+        ...
+        The message you want to broadcast or mirror
+        (with it's own "key" and "data" fields)
+        ...
+    }
+}
+\end{lstlisting}
+Any invalid messages (not valid JSON or no \texttt{key} field present) are ignored
+by the TFW server.
+
+\subsection{Event Handlers}
+
+Event handlers are processes running in the \texttt{solvable} Docker container
+that subscribe to specific message types using ZeroMQ sockets.
+As discussed earlier, using ZeroMQ allows developers to implement event handlers
+in a wide variety of programming languages.
+This is very important for the framework, as content creators often create
+challenges that are very specific to a language, for example the showcasing
+of a security vulnerability in an older version of Java.
+
+These event handlers are used to write most of the code developers wish to
+integrate with the framework.
+Let's say that a tutorial asks the user to fix some piece of legacy C code.
+In this case, a challenge developer could implement an event handler that runs
+some unit tests to determine whether the user was successful in fixing
+the code or not, then advance the tutorial or invoke other event handlers
+based on this knowledge.
+An event handler such as this could be invoked by sending a message to it
+at any time when the running of the tests would be required.
+
 \subsection{Frontend}
 
 This is a web application that runs in the browser of the user and uses
@@ -152,6 +238,138 @@ which tries to remedy the issues\cite{JavaScript}
 with JavaScript by being a language that transpiles to JavaScript while
 strongly encouraging things like static typing or Object Oriented Principles.
 
-\subsection{Messaging}
+\pic{figures/tfw_frontend.png}{The Current Angular Frontend of the Tutorial Framework}
+
+\subsection{Built-in Event Handlers and Frontend Components}
+
+A good chunk of the framework codebase is a bunch of pre-made, built-in components
+that implement commonly required functionality for developers to use.
+These components usually involve an event handler and an Angular component which
+communicates with it to realize some functionality.
+An example would be the built-in code editor of the framework
+(visible on the left side of Figure~\ref{figures/tfw_frontend.png}).
+This code editor is essentially a Monaco editor%
+\footnote{\href{https://microsoft.github.io/monaco-editor/}
+{https://microsoft.github.io/monaco-editor/}}
+instance integrated into Angular and upgraded with the capability to
+exchanges messages with an event handler to save, read and edit files
+that reside in the writeable file system of the \texttt{solvable}
+Docker container.
+
+All of the built-ins come with full API documentation explaining what they do
+on receiving specific messages, and what messages they emit on different events.
+This greatly expands the capabilities of the framework, since it allows
+developers to do things including, but not limited to:
+\begin{itemize}
+    \item making the code editor automatically appear in sections
+          of the tutorial where the user needs to use it
+    \item inject commands into the user's terminal
+    \item hook into messages emitted from components to detect events, such as
+          to detect if the user has clicked a button or executed a command
+          in the terminal
+    \item monitor the logs (stdout or stderr) of a given process
+\end{itemize}
+Every pre-made component is designed with the mindset to allow flexible
+and creative usage by developers, with the possibility of future extensions.
+Often when developers require certain new features, they open an issue on
+the git repository of the framework for me to review and possibly implement
+later.
+
+One example would be when a developer wanted to automatically advance the tutorial
+when the user has entered a specific string into a file.
+This one didn't even require a new feature: I recommended him to implement an event
+handler listening to the messages of the built-in file editor, filter the messages
+which contain file content that is going to be written to disk, and simply
+search these messages for the given string.
+
+The exact capabilities of these built-in components will be explained in greater
+detail in a later chapter.
+
 \subsection{TFW Finite State Machine}
-\subsection{Solution Checking}\label{solutioncheck}
+
+An important requirement we have specified during~\ref{requirements} was that
+the framework must be capable of tracking user progress.
+TFW allows developers to define a \emph{finite state machine}
+which is capable of describing the desired ``story'' of a tutorial.
+The states of the machine could be certain points in time during the completion of the
+tutorial envisioned and transitions could be events that influence the
+state, such as the editing of files, execution of commands and so on.
+
+Take the fixing of a SQL Injection%
+\footnote{\href{https://www.owasp.org/index.php/SQL_Injection}
+{https://www.owasp.org/index.php/SQL\_Injection}}
+vulnerability as an example.
+Let's assume, that the source code is vulnerable to a SQL injection attack
+because it tries to compose a query with string concatenation instead of
+using a parameterized query provided by the database library.
+A challenge developer could implement an FSM in the framework that looks like this:
+
+\pic[width=.6\textwidth]{figures/tfw_fsm.png}{An Example for a Finite State Machine in TFW}
+
+In case the source file has been edited, the unit test cases designed to detect
+whether the code is vulnerable or not are invoked.
+Depending on the results three cases are possible:
+
+\begin{description}
+    \item[All test cases have succeeded:] If all the tests succeeded then the user has managed
+    to fix the code properly and we can display a congratulating message accordingly.
+    \item[All test cases have failed:] In this case the solution is incorrect
+    and we can offer some hints.
+    \item[Some test cases have succeeded:] It is possible that the based on the test cases
+    that have succeeded and failed we can determine that the user tried to blacklist
+    certain SQL keywords. This is a common, but incorrect solution of fixing a SQL
+    injection vulnerability. Now we can explain to users why their solution is wrong,
+    and give them helpful tips.
+\end{description}
+
+This example shows how content creators can create tutorials that could behave
+in many different ways based on what the user does.
+In high quality challenges developers can implement several ``paths'' to
+a successful completion.
+This is a very engaging feature that offers an immersive learning experience for
+users, which many solutions for distance education lack so often.
+
+Developers can use a YAML file or write Python code to implement finite
+state machines.
+Architecturally a built-in event handler called \texttt{FSMManagingEventHandler}
+is capable of managing the FSM defined by clients.
+The responsibilities of said event handler include:
+\begin{itemize}
+    \item Attempting to step the state machine (one can write preconditions that are
+    required to succeed for the transition to take place)
+    \item Broadcasting FMS update messages on events involving the state machine,
+    such as successful transitions
+\end{itemize}
+
+The \texttt{trigger} field of a message can be used to step the framework FSM
+if all preconditions are met.
+The way this works is if the TFW server encounters a message with a
+\texttt{trigger} defined, it notifies the event handler managing
+the state machine.
+
+Since messages can come from unauthenticated sources, it is possible to
+enforce the authentication of privileged messages, such as messages containing a \texttt{trigger}.
+The framework allows trusted code to access a cryptographic key on the file system, which
+can be used to digitally sign messages (this is what the \texttt{signature} message
+field is designed for).
+In this case the TFW server will only forward privileged messages that
+have a valid signature.
+
+\subsection{Solution checking}
+
+Traditionally most challenges on the Avatao platform implement a Docker image called
+\texttt{controller}, which is responsible for detecting the successful
+solution of a challenge.
+When using the Tutorial Framework a pre-implemented \texttt{controller}
+image is available, which listens to messages emitted by the
+framework FSM, and detects if the final state defined by developers is reached.
+This means that if content creators implement a proper FSM, the solution checking
+does not require any more effort from their part and will work automatically.
+
+It is also worth to note that the authentication of privileged messages
+makes the Tutorial Framework suitable for implementing
+traditional hacking challenges, such as exercises developed for CTF%
+\footnote{A ``capture the flag'' game is a competition designed for professionals
+--- or just people interested in the field ---  to sharpen their skills in IT security.
+Avatao often organises similar events.}
+events.

latexplate.sty

@@ -27,5 +27,6 @@
     \centering
     \includegraphics[#1]{#2}
     \caption{#3}
+    \label{#2}
 \end{figure}
 }

thesis.tex

@@ -30,8 +30,6 @@
 \pagestyle{fancy}
 \fancyfoot[C]{\thepage}
 
-\newcommand{\crule}{\rule{\textwidth}{1px}\\\\}
-
 \begin{document}
 \include{content/titlepage}
 
@@ -39,7 +37,7 @@
 \tableofcontents
 \microtypesetup{protrusion=true}
 \thispagestyle{empty}
-\cleardoublepage
+\cleardoublepage{}
 
 \include{content/declaration}
 \include{content/abstract}
@@ -47,7 +45,7 @@
 \include{content/architecture}
 
 \listoffigures
-\lstlistoflistings
+\lstlistoflistings{}
 
 \renewcommand\bibname{References}
 \printbibliography{}