diff --git a/content/architecture.tex b/content/architecture.tex index dfbbf0f..4cccf0b 100644 --- a/content/architecture.tex +++ b/content/architecture.tex @@ -124,6 +124,92 @@ greater detail, as well as how they interact with each other, their respective responsibilities, some of the design choices behind them and more. +\subsection{TFW Message Format} + +All components in the Tutorial Framework use JSON% +\footnote{JavaScript Object Notation: \href{https://www.json.org}{https://www.json.org}} +messages to communicate with eachother. +These messages must also comply some simple rules specific to the framework. +Let's inspect further what a valid TFW message might look like: + +\begin{lstlisting}[captionpos=b,caption={The TFW JSON message format}] +{ + "key": ...an identifier used for addressing..., + "data": + { + ... + optional JSON object carrying arbitrary data + ... + }, + "trigger": ...optional state change action..., + "signature": ...optional HMAC signature for authenticated messages..., + "seq": ...sequence number that is automatically inserted by the TFW server... +} +\end{lstlisting} + +All valid messages \emph{must} include a \texttt{key} field as this is used by the +framework for addressing: event handlers and frontend components subscribe to one +or more \texttt{key}s and only receive messages with \texttt{key}s they have +subscribed to. +It is possible to send a message with an empty key, however these messages will not +be forwarded by the TFW server (but will reach it, so in case the target of a message +is the TFW server exclusively, an empty \texttt{key} can may used). + +The rest of the fields are optional, but most messages will carry something +in their \texttt{data} field. +The purpose \texttt{trigger} and \texttt{signature} fields will be detailed +at a later point in this paper. + +\subsection{Networking Details} + +The default behaviour of the TFW server is that it forwards all messages from coming from +the frontend to the event handlers and vice versa. +So messages coming from the WebSockets of the frontend are forwarded to event handlers +via ZMQ and messages received through ZMQ from event handlers are forwarded to +the frontend via WebSockets. + +The TFW server is also capable of ``reflecting'' messages back to the side they were +received on (to faciliate event handler to event handler for instance), or broadcast +messages to all components. +This is possible by embedding a whole TFW message in the \texttt{data} field of +an outer wrapper message with a special \texttt{key} that signals to the TFW server that +this message requires special attention. +An example of this would be: +\begin{lstlisting}[captionpos=b,caption={Broadcasting and mirroring TFW messages}] +} + "key": "broadcast", // or "mirror" + "data": + { + ... + The message you want to broadcast or mirror + (with it's own "key" and "data" fields) + ... + } +} +\end{lstlisting} +Any invalid messages (not valid JSON or no \texttt{key} field present) are ignored +by the TFW server. + +\subsection{Event Handlers} + +Event handlers are processes running in the \texttt{solvable} Docker container +that subscribe to specific message types using ZeroMQ sockets. +As discussed earlier, using ZeroMQ allows developers to implement event handlers +in a wide variety of programming languages. +This is very important for the framework, as content creators often create +challenges that are very specific to a language, for example the showcasing +of a security vulnerability in an older version of Java. + +These event handlers are used to write most of the code developers wish to +integrate with the framework. +Let's say that a tutorial asks the user to fix some piece of legacy C code. +In this case, a challenge developer could implement an event handler that runs +some unit tests to determine whether the user was successful in fixing +the code or not, then advance the tutorial or invoke other event handlers +based on this knowledge. +An event handler such as this could be invoked by sending a message to it +at any time when the running of the tests would be required. + \subsection{Frontend} This is a web application that runs in the browser of the user and uses @@ -152,6 +238,138 @@ which tries to remedy the issues\cite{JavaScript} with JavaScript by being a language that transpiles to JavaScript while strongly encouraging things like static typing or Object Oriented Principles. -\subsection{Messaging} +\pic{figures/tfw_frontend.png}{The Current Angular Frontend of the Tutorial Framework} + +\subsection{Built-in Event Handlers and Frontend Components} + +A good chunk of the framework codebase is a bunch of pre-made, built-in components +that implement commonly required functionality for developers to use. +These components usually involve an event handler and an Angular component which +communicates with it to realize some functionality. +An example would be the built-in code editor of the framework +(visible on the left side of Figure~\ref{figures/tfw_frontend.png}). +This code editor is essentially a Monaco editor% +\footnote{\href{https://microsoft.github.io/monaco-editor/} +{https://microsoft.github.io/monaco-editor/}} +instance integrated into Angular and upgraded with the capability to +exchanges messages with an event handler to save, read and edit files +that reside in the writeable file system of the \texttt{solvable} +Docker container. + +All of the built-ins come with full API documentation explaining what they do +on receiving specific messages, and what messages they emit on different events. +This greatly expands the capabilities of the framework, since it allows +developers to do things including, but not limited to: +\begin{itemize} + \item making the code editor automatically appear in sections + of the tutorial where the user needs to use it + \item inject commands into the user's terminal + \item hook into messages emitted from components to detect events, such as + to detect if the user has clicked a button or executed a command + in the terminal + \item monitor the logs (stdout or stderr) of a given process +\end{itemize} +Every pre-made component is designed with the mindset to allow flexible +and creative usage by developers, with the possibility of future extensions. +Often when developers require certain new features, they open an issue on +the git repository of the framework for me to review and possibly implement +later. + +One example would be when a developer wanted to automatically advance the tutorial +when the user has entered a specific string into a file. +This one didn't even require a new feature: I recommended him to implement an event +handler listening to the messages of the built-in file editor, filter the messages +which contain file content that is going to be written to disk, and simply +search these messages for the given string. + +The exact capabilities of these built-in components will be explained in greater +detail in a later chapter. + \subsection{TFW Finite State Machine} -\subsection{Solution Checking}\label{solutioncheck} + +An important requirement we have specified during~\ref{requirements} was that +the framework must be capable of tracking user progress. +TFW allows developers to define a \emph{finite state machine} +which is capable of describing the desired ``story'' of a tutorial. +The states of the machine could be certain points in time during the completion of the +tutorial envisioned and transitions could be events that influence the +state, such as the editing of files, execution of commands and so on. + +Take the fixing of a SQL Injection% +\footnote{\href{https://www.owasp.org/index.php/SQL_Injection} +{https://www.owasp.org/index.php/SQL\_Injection}} +vulnerability as an example. +Let's assume, that the source code is vulnerable to a SQL injection attack +because it tries to compose a query with string concatenation instead of +using a parameterized query provided by the database library. +A challenge developer could implement an FSM in the framework that looks like this: + +\pic[width=.6\textwidth]{figures/tfw_fsm.png}{An Example for a Finite State Machine in TFW} + +In case the source file has been edited, the unit test cases designed to detect +whether the code is vulnerable or not are invoked. +Depending on the results three cases are possible: + +\begin{description} + \item[All test cases have succeeded:] If all the tests succeeded then the user has managed + to fix the code properly and we can display a congratulating message accordingly. + \item[All test cases have failed:] In this case the solution is incorrect + and we can offer some hints. + \item[Some test cases have succeeded:] It is possible that the based on the test cases + that have succeeded and failed we can determine that the user tried to blacklist + certain SQL keywords. This is a common, but incorrect solution of fixing a SQL + injection vulnerability. Now we can explain to users why their solution is wrong, + and give them helpful tips. +\end{description} + +This example shows how content creators can create tutorials that could behave +in many different ways based on what the user does. +In high quality challenges developers can implement several ``paths'' to +a successful completion. +This is a very engaging feature that offers an immersive learning experience for +users, which many solutions for distance education lack so often. + +Developers can use a YAML file or write Python code to implement finite +state machines. +Architecturally a built-in event handler called \texttt{FSMManagingEventHandler} +is capable of managing the FSM defined by clients. +The responsibilities of said event handler include: +\begin{itemize} + \item Attempting to step the state machine (one can write preconditions that are + required to succeed for the transition to take place) + \item Broadcasting FMS update messages on events involving the state machine, + such as successful transitions +\end{itemize} + +The \texttt{trigger} field of a message can be used to step the framework FSM +if all preconditions are met. +The way this works is if the TFW server encounters a message with a +\texttt{trigger} defined, it notifies the event handler managing +the state machine. + +Since messages can come from unauthenticated sources, it is possible to +enforce the authentication of privileged messages, such as messages containing a \texttt{trigger}. +The framework allows trusted code to access a cryptographic key on the file system, which +can be used to digitally sign messages (this is what the \texttt{signature} message +field is designed for). +In this case the TFW server will only forward privileged messages that +have a valid signature. + +\subsection{Solution checking} + +Traditionally most challenges on the Avatao platform implement a Docker image called +\texttt{controller}, which is responsible for detecting the successful +solution of a challenge. +When using the Tutorial Framework a pre-implemented \texttt{controller} +image is available, which listens to messages emitted by the +framework FSM, and detects if the final state defined by developers is reached. +This means that if content creators implement a proper FSM, the solution checking +does not require any more effort from their part and will work automatically. + +It is also worth to note that the authentication of privileged messages +makes the Tutorial Framework suitable for implementing +traditional hacking challenges, such as exercises developed for CTF% +\footnote{A ``capture the flag'' game is a competition designed for professionals +--- or just people interested in the field --- to sharpen their skills in IT security. +Avatao often organises similar events.} +events. diff --git a/figures/tfw_frontend.png b/figures/tfw_frontend.png new file mode 100644 index 0000000..c59799e Binary files /dev/null and b/figures/tfw_frontend.png differ diff --git a/figures/tfw_fsm.png b/figures/tfw_fsm.png new file mode 100644 index 0000000..09ca27b Binary files /dev/null and b/figures/tfw_fsm.png differ diff --git a/latexplate.sty b/latexplate.sty index 63eb75e..3bfc578 100644 --- a/latexplate.sty +++ b/latexplate.sty @@ -27,5 +27,6 @@ \centering \includegraphics[#1]{#2} \caption{#3} + \label{#2} \end{figure} } diff --git a/thesis.tex b/thesis.tex index 26074b9..26c3b45 100644 --- a/thesis.tex +++ b/thesis.tex @@ -30,8 +30,6 @@ \pagestyle{fancy} \fancyfoot[C]{\thepage} -\newcommand{\crule}{\rule{\textwidth}{1px}\\\\} - \begin{document} \include{content/titlepage} @@ -39,7 +37,7 @@ \tableofcontents \microtypesetup{protrusion=true} \thispagestyle{empty} -\cleardoublepage +\cleardoublepage{} \include{content/declaration} \include{content/abstract} @@ -47,7 +45,7 @@ \include{content/architecture} \listoffigures -\lstlistoflistings +\lstlistoflistings{} \renewcommand\bibname{References} \printbibliography{}