thesis/content/using_the_framework.tex

\chapter{Using the Framework}\label{usingtfw}

In this section I am going to dive into further detail on how client code is supposed
to use the framework, some of the design decisions behind this and how everything is
is integrated into the \code{solvable} Docker image.

To use the framework one has to do several things to get started.
The main points include:
\begin{itemize}
    \item Setting up a development environment
    \item Defining an FSM to describe the flow of the tutorial and implementing proper callbacks
          for this machine, such as ones that display messages to the user
    \item Implementing the required event handlers, which may trigger state transitions in the FSM,
          interact with non-TFW code and do various things that might be needed during an exercise,
          such as compiling code written by the user or running unit tests
    \item Defining what processes should run inside the container besides the things TFW
          starts automatically
    \item Setting up reverse proxying for any user-facing network application such as web servers
\end{itemize}
At first all these tasks can seem quite overwhelming.
Remember that \emph{witchcraft} is what we practice here after all.
To overcome the high initial learning curve of getting familiar with the framework
I have created a repository called \emph{test-tutorial-framework} with the purpose of
providing a project template for developers looking to create challenges using the
framework.
This repository is a really simple client codebase that is suitable for
developing TFW itself as well (a good place to host tests for the framework).

It also provides an ``industry standard'' \code{hack} directory
containing bash scripts that make the development of tutorials and TFW itself very convenient.
These scripts span from bootstrapping a complete development environment in one command,
to building and running challenges based on the framework. 
Let us take a quick look at the \emph{test-tutorial-framework} project to acquire a greater
understanding of how the framework interacts with client code.

\section{Project Structure}

\begin{lstlisting}[
    caption={The project structure of test-tutorial-framework},
    captionpos=b]
.
|--config.yml
|
|--hack/
|   |--tfw.sh
|   |--...
|
|--controller/
|   |--Dockerfile
|   |--...
|
|--solvable/
    |--Dockerfile
    |--...
\end{lstlisting}

\subsection{Avatao Specific Files}
The \code{config.yml} file is an Avatao challenge configuration file,
which is used describe what kind of Docker containers implement a challenge,
what ports do they expose talking what protocols, define the name of the
exercise, it's difficulty, and so on.
Every Avatao challenge must provide such a file.
Another thing that is not even indicated on the structure above is the \code{metadata}
directory, which contains the short and long descriptions of challenges in
Markdown format.

The Tutorial Framework does not use these files in any way whatsoever,
these are only required to make the tutorial function on the Avatao platform.

\subsection{Controller Image}
It was previously mentioned that the \code{controller} Docker image is responsible
for the solution checking of challenges (whether the user has completed the exercise or not).
Currently this image is maintained in the test-tutorial-framework repository.
It is a really simple Python server which functions as a TFW event handler as well.
It subscribes to the FSM update messages
broadcasted by the \code{FSMManagingEventHandler}, we have discussed previously,
this way it is capable of keeping track of the state of the tutorial,
which allows it to detect if the final state of the FSM is reached.

\subsection{Solvable Image}
Currently the Tutorial Framework is maintained in three git repositories:
\begin{description}
      \item[baseimage-tutorial-framework:] Docker baseimage (contains all backend logic)
      \item[frontend-tutorial-framework:] Angular frontend
      \item[test-tutorial-framework:] An example tutorial built using baseimage and frontend
\end{description}
Every tutorial based on the framework must use the TFW baseimage as the parent of
it's own \code{solvable} image, using the \code{FROM}%
\footnote{\href{https://docs.docker.com/engine/reference/builder/\#from}
{https://docs.docker.com/engine/reference/builder/\#from}}
Dockerfile command.
Being an example project of the framework this is the case with
test-tutorial-framework as well.

\section{Details of the Solvable Image}
Let us dive into greater detail on how the \code{solvable} Docker image of the
test-tutorial-framework operates.
The directory structure is as follows:
\begin{lstlisting}
solvable/
|--Dockerfile
|--frontend/
|--supervisor/
|--nginx/
|--src/
\end{lstlisting}
I am going to discuss these one by one.

\subsection{Dockerfile}
Since this is a Docker image it must define a \code{Dockerfile}.
This image always uses the baseimage of the framework as a parent image.
Besides this developers can use this as a regular \code{Dockerfile} to work with
in any way they see fit to implement their tutorial.
This means that developers looking to create content on Avatao, be that
with the Tutorial Framework or without it must be familiar with Docker,
as they will have to set everything up to work inside a container.

\subsection{Frontend}
This directory is designed to contain a clone of the frontend repository.
By default it is empty and it's contents will be put in place during the
setup of the development environment.

\subsection{Supervisor}
As previously mentioned, the framework uses supervisor to run several processes
inside a Docker container.
Usually Docker containers only run a single process and developers simply start
more containers instead of processes if required (and use tools such as docker-compose%
\footnote{\href{https://docs.docker.com/compose/}{https://docs.docker.com/compose/}}
or kubernetes%
\footnote{\href{https://kubernetes.io}{https://kubernetes.io}}
to orchestrate their containers).
This approach is not suitable for TFW, as it would require the framework to orchestrate
Docker containers from inside a container managed by the same Docker daemon, which is
feasible in theory but very hard and unserviceable to do in practice.
This would require doing something like mounting the UNIX domain socket used
to manage the Docker daemon inside a running container managed by that daemon,
which is a fun thing to
play around with in my free time but not something suitable for running in production,
not even mentioning the severe security implications of doing something like that.

Supervisor is a process control system designed to be able to work with
processes on UNIX-like operating systems.
When a tutorial built on TFW is started, a Docker container starts with supervisor running as
PID\footnote{Process ID, on UNIX-like systems the \code{init} program is the first
process started, and who gets PID 1 traditionally.} 1, which in turn starts all the
programs defined in the \code{solvable/supervisor} directory.
Content creators can use supervisor configuration files to define these programs.
For example, a developer would write a file similar to this one and place it into the
\code{solvable/supervisor} directory to run a web server written in Python:
\begin{lstlisting}
[program:yourprogram]
user=user
directory=/home/user/example/
command=python3 server.py
autostart=true
\end{lstlisting}
As mentioned earlier in~\ref{processmanagement}, any program that is started this way
can be managed by the framework using API messages.
All this is possible through using the xmlrpc%
\footnote{\href{https://docs.python.org/3/library/xmlrpc.html}
{https://docs.python.org/3/library/xmlrpc.html}}
API exposed by supervisor, which allows the framework to interact with it to control processes.
This API is quite flexible and can be used to achieve a number of things which would be
clumsy to do without using it (i.e.\ supervisor has a command line utility called
\code{supervisorctl} that exposes similar functionality to the xmlrpc bindings,
but it is better to communicate with the supervisor daemon directly than to
invoke it's command line utility in a separate process when you need something done).

\subsection{Nginx}
For simplicity, exercises based on the framework only expose a single port from the
\code{solvable} container.
This port is required to serve the frontend of the framework.
If this is the case, how do we run additional web applications to showcase vulnerabilities
on during a tutorial?
Since one port can only be bound by one process at a time, we will need to
run a reverse-proxy%
\footnote{\href{https://www.nginx.com/resources/glossary/reverse-proxy-server/}
{https://www.nginx.com/resources/glossary/reverse-proxy-server/}} server inside the
container to
bind the exposed port and redirect traffic to other web servers binding non-exposed ports.

To support this, TFW automatically starts an nginx web server. It uses this nginx
instance to serve the framework frontend as well.
It is possible to supply additional configurations to this server in a convenient manner:
any configuration files placed into the \code{solvable/nginx} directory will be
interpreted by nginx once the container has started.
To set up the reverse-proxying of a web server running on port 3333,
one would write a configuration file similar to this one:
\begin{lstlisting}
location /yoururl {
    proxy_pass http://127.0.0.1:3333;
}
\end{lstlisting}
Now the content served by this web server on port 3333
will be available on the URL \code{<challenge-url>/yoururl} despite that port 3333
does not accept connections from outside the container as it is not exposed.
It is very important to understand, that developers
have to make sure that their web application \emph{behaves well} behind a reverse proxy.
What this means is that they are going to be served from a ``subdirectory'' of the top
level URL\@:
for example \code{/register} will be served under \code{/yoururl/register}.
This means that all links in the final HTML must refer to the proxied URLs, e.g.\
\code{/yoururl/login}, and server-side redirects must point to these correct hrefs as well.
Idiomatically this is usually implemented by supplying a \code{BASEURL}
to the application through an environment variable, so that it is able to set
itself up correctly.

\subsection{Copying Configuration Files}
Behind the curtains, the Tutorial Framework uses some Dockerfile trickery to
facilitate the copying of supervisor and nginx configuration files to their correct
locations.
Normally when one uses the \code{COPY}%
\footnote{\href{https://docs.docker.com/engine/reference/builder/\#copy}
{https://docs.docker.com/engine/reference/builder/\#copy}}
command to create a layer%
\footnote{\href{https://docs.docker.com/storage/storagedriver/}
{https://docs.docker.com/storage/storagedriver/}} in a Docker image,
this action takes place on building that image (i.e.\ in the \emph{build context}
of that image).
This is not good for this use case: when building the framework baseimage,
these configuration files that will be written by content developers using TFW in
the future do not even exist yet.
How could we copy files into an image layer that will be created in the future?

It is possible to use a command called \code{ONBUILD}%
\footnote{\href{https://docs.docker.com/engine/reference/builder/\#onbuild}
{https://docs.docker.com/engine/reference/builder/\#onbuild}}
in the Dockerfile of a baseimage to delay another command
to the point in time where other images will use the baseimage
as a parent with the \code{FROM} command. This makes it possible to execute
commands in the build context of the descendant image.
This is great, because the config files we need \emph{will} exist in the build
context of the \code{solvable} image of test-tutorial-framework.
In practice this looks something like this in the baseimage \code{Dockerfile}:
\begin{lstlisting}
ONBUILD COPY ${BUILD_CONTEXT}/nginx/ ${TFW_NGINX_COMPONENTS}
ONBUILD COPY ${BUILD_CONTEXT}/supervisor/ ${TFW_SUPERVISORD_COMPONENTS}
\end{lstlisting}
It is important to keep in mind however, that the layers created by these
\code{ONBUILD} commands will only be available \emph{after} the \code{FROM}
command is executed when building the child image \emph{in the future}.
This means that if you want to
do something with these files in the baseimage build after they have
been copied, those things must be done in \code{ONBUILD} commands as well.

\subsection{Source Directory}
The \code{src} directory usually holds tutorial-specific code, such as
the implementations of event handlers, the framework FSM, additional web applications
served by the exercise and generally anything that won't fit in the other,
framework-specific directories.
The use of this directory is not mandatory, only a good practice, as developers
are free to implement the non-TFW parts of their exercises as they see fit
(the copying of these files into image layers using \code{solvable/Dockerfile}
is their responsibility as well).

\section{Configuring Built-in Components}

The configuration of built-ins is generally done in two different ways.
For the frontend (Angular) side, developers can edit a \code{config.ts} file,
which is full of key-value pairs of configurable frontend functionality.
These pairs are generally pretty self-documenting:
\lstinputlisting[
    caption={Example of the frontend \code{config.ts} file (shortened to save space)},
    captionpos=b
]{listings/config.ts}
Configuring built-in event handlers is possible by editing the Python file they are
initialized in, which exposes several communicative options through the
\code{__init__()} methods of these event handlers:
\lstinputlisting[
    language=python,
    caption={Example of initializing some event handlers},
    captionpos=b
]{listings/event_handler_main.py}

\section{Implementing a Finite State Machine}

The Tutorial Framework allows developers to define state machines in two ways,
as discussed before.
I am going to display the implementation of the same FSM using these methods
to showcase the capabilities of the framework.

\subsection{YAML based FSM}
YAML\footnote{YAML Ain't Markup Language: \href{http://yaml.org}{http://yaml.org}}
is a human friendly data serialization standard and a superset of JSON\@.
It is possible to use this format to define a state machine like so:
\lstinputlisting[
    caption={A Finite State Machine implemented in YAML},
    captionpos=b
]{listings/test_fsm.yml}
This state machine has two states, state \code{0} and \code{1}.
It defines a single transition between them, \code{step_1}.
On entering state \code{1} the FSM will write a message to the frontend messaging component
by invoking TFW library code using Python.
The transition can only occur if the file \code{allow_step_1} exists.

YAML based state machine implementations also allow the usage of the Jinja2%
\footnote{\href{http://jinja.pocoo.org/docs/2.10/}{http://jinja.pocoo.org/docs/2.10/}}
templating language to substitute variables into the YAML file.
These substitutions are really powerful, as one could even iterate through arrays
or invoke functions that produce strings to be inserted using this method.
This is very similar to how Ansible uses%
\footnote{\href{https://docs.ansible.com/ansible/2.6/user_guide/playbooks_templating.html}
{https://docs.ansible.com/ansible/2.6/user\_guide/playbooks\_templating.html}}
Jinja2, and I was certainly inspired by this
when coming up with this idea.
For example, if we had an FSM with five states, we could use the following
Jinja2 code to generate a transition called \code{step_next} between each state
in a \code{for} cycle:
\begin{lstlisting}
{% for i in range(5) %}
-   trigger: 'step_next'
    source: '{{i}}'
    dest: '{{i+1}}'
{% endfor %}
\end{lstlisting}

\subsection{Python based FSM}
Optionally, the same state machine can be implemented like this in Python using
TFW library code:
\lstinputlisting[
    language=python,
    caption={A Finite State Machine implemented in Python},
    captionpos=b
]{listings/test_fsm.py}

As you can see, both implementations are pretty clean and easy.
The advantage of YAML is that it makes it possible to define callbacks using virtually any
command that is available in the container, which means any
programming language is usable to implement said callbacks.
The advantage of the Python version is that since the framework is being developed in
Python as well, it is going to be easier to interface with library code.

\section{Implementing Event Handlers}

In this section I am going to showcase how implementing event handlers is possible
when using the framework.
I am going to use the Python programming language, but it isn't hard
to create event handlers in other languages, as the only thing
they have to be capable of is communicating with the TFW server using
ZeroMQ sockets, as previously discussed.
The library provided by the framework abstracts low-level socket logic
away by implementing easy to use base classes.
Creating such base classes in a given language shouldn't take longer
than a few hours for an experienced developer.
Our challenge creators have already implemented similar libraries for
Java, JavaScript and C++ as well.
\lstinputlisting[
    language=python,
    caption={A very simple event handler implemented in Python},
    captionpos=b
]{listings/event_handler_example.py}
This simple event handler subscribes to the \code{fsm_update} messages,
then the only thing it does is writing the
messages received to the messages component on the frontend.
When using the TFW library in Python, all classes inheriting from
\code{EventHandlerBase} must implement the \code{handle_event()}
abstract method, which is used to, well, handle events.

\section{Setting Up a Developer Environment}\label{devenv}

To make getting started as smooth as possible I have created
a ``bootstrap'' script which is capable of creating a development environment from
scratch.
This script is distributed as the following bash one-liner:
\begin{lstlisting}[language=bash]
bash -c "$(curl -fsSL https://git.io/vxBfj)"
\end{lstlisting}
This command downloads the script using \code{curl}%
\footnote{\href{https://curl.haxx.se}{https://curl.haxx.se}}, then executes it in bash.
In the open source community it is quite common to distribute installers this way%
\footnote{A good example of this is oh-my-zsh:
\href{https://github.com/robbyrussell/oh-my-zsh}{https://github.com/robbyrussell/oh-my-zsh}},
which might seem a little scary at first, but is not less safe than
downloading and executing a binary installer from a website with a valid TLS certificate, as
\code{curl} will fail with an error message if the certificate is invalid.
This is because both methods place their trust in the PKI~\footnote{Public Key Infrastructure}
to defend against man-in-the-middle%
\footnote{\href{https://www.owasp.org/index.php/Man-in-the-middle_attack}
{https://www.owasp.org/index.php/Man-in-the-middle\_attack}} attacks.
Debating the security of this infrastructure is certainly something that we
as an industry should constantly do, but it is out of the scope of this paper.

Nevertheless I have also created a version of this command that
checks the SHA256 checksum of the bootstrap script before executing it
(I have placed it on several lines to enhance visibility):
\begin{lstlisting}[language=bash]
URL=https://git.io/vxBfj                                             \
SHA=d81057610588e16666251a4167f05841fc8b66ccd6988490c1a2d2deb6de8ffa \
bash -c 'cmd="$(curl -fsSL $URL)" &&                                 \
         [ $(echo "$cmd" | sha256sum | cut -d " " -f1) == $SHA ] &&  \
         echo "$cmd" | bash || echo Checksum mismatch!'
\end{lstlisting}
This essentially downloads the script, places it inside a variable as a string,
then pipes it into a bash interpreter \emph{only if} the checksum
of the downloaded string matches the one provided, otherwise it displays
an error message.
Software projects distributing their product as binary installers often
display such checksums on their download pages with the purpose of potentially
mitigating MITM attacks.

The bootstrap script clones the three TFW repositories and does several steps
to create a working environment into a single directory, that is based on
test-tutorial-framework:
\begin{itemize}
      \item It builds the newest version of the TFW baseimage locally
      \item It pins the version tag of this image in \code{solvable/Dockerfile},
            so that this newly-built version will be used by the tutorial
      \item It places the latest frontend in \code{solvable/frontend} with
            dependencies installed
\end{itemize}
It is important to note that this script \emph{does not} install anything system-wide,
it only works in the directory it is being executed from.
This is a good practice, as many users --- including me --- find scripts that
write files all around the system intrusive if they could provide the same functionality
while working in a single directory.

It is also worth to mention that it would have been a lot easier to simply use Docker Hub%
\footnote{\href{https://hub.docker.com}{https://hub.docker.com}},
but since the code base is currently proprietary we cannot distribute
it using a public medium, and we use our own image registry to store private Docker
images.

\section{Building and Running a Tutorial}

After the environment has been created using the script described in~\ref{devenv},
it is possible to simply use standard Docker commands to build and run the tutorial.
Yet the \code{hack} directory of test-TFW also provides a script called
\code{tfw.sh} that developers prefer to use for building and running their
exercises.
Why is this the case?

\subsection{The Frontend Issue}

To be able to understand this, we will have to gain some understanding on how the
build process of Angular projects work.

When frontend developers work on Angular projects, they usually use the built-in
developer tool of the Angular-CLI%
\footnote{\href{https://cli.angular.io}{https://cli.angular.io}},
\code{ng serve} to build and serve their applications.
The advantage of this tool is that it automatically reloads the frontend
when the code on the disk is changed, and that it is generally very easy to work with.
On the other hand, a disadvantage is that a \code{node_modules} directory
containing all the npm%
\footnote{\href{https://www.npmjs.com}{https://www.npmjs.com}}
dependencies of the project must be present while doing so.
The problem with this is that because the JavaScript ecosystem is a \emph{huge}
mess\cite{NodeModules}, these dependencies can easily get up to
\emph{several hundreds of megabytes} in size.

To solve this issue, when creating production builds,
Angular uses various optimizations such as tree shaking%
\footnote{\href{https://webpack.js.org/guides/tree-shaking/}
{https://webpack.js.org/guides/tree-shaking/}}
to remove all the dependencies that won't be used when running the application%
\footnote{Otherwise it won't be possible to serve these applications efficiently
over the internet.}.
The problem is, that these things can take a \emph{really} long time.
This is why today frontend builds usually take a lot longer then building anything
not involving JavaScript (such as C++, C\# or any other compiled programming language).

This mess presents it's own challenges for the Tutorial Framework as well.
Since hundreds of megabytes of npm dependencies have no place inside Docker images%
\footnote{Or it may take tens of seconds just to send the build context to
the Docker daemon, which means waiting even before the build began.},
by default the framework will only copy the results of a frontend production build
of \code{solvable/frontend} into the image layers.
This slows down the build time of TFW based challenges so much, that instead of like
30 seconds, they could often take 5 to 10 minutes depending on what hardware
you use.

\subsection{The Solution Offered by the Framework}

To circumvent this, it is possible to entirely exclude the Angular frontend from a TFW
build, using build time arguments%
\footnote{In practice this is done by supplying the option
\code{--build-arg NOFRONTEND=1} to Docker.}.
But when doing so, developers would have to run the frontend locally with
the whole \code{node_modules} directory present.
The bootstrap script takes care of putting these dependencies there,
while the \code{tfw.sh} script is capable of starting a development server
to serve the frontend locally using \code{ng serve} besides starting
the Docker container without the frontend.
If this whole thing wasn't complicated enough, since Docker binds the port
the container is going to use, \code{tfw.sh} has to run the Angular dev server on
another port, then use the proxying features of Angular-CLI to forward requests
from this port to the running Docker container when requesting resources
other than the entrypoint to the Angular application.

This is the reason why the frontend is accessible through port \code{4200} (default
port for \code{ng serve}) when using \code{tfw.sh} to start a tutorial, but when running
a self-contained container built with the frontend included it is accessible on port \code{8888}
(the default port TFW uses).

While it also provides lots of other functionality, this is one of the reasons why
the \code{tfw.sh} script is a several hundreds of lines long bash script.
The implementation of making the frontend toggleable during Docker builds requires some
of the \code{ONBUILD} stuff we've discussed earlier:
\begin{lstlisting}[language=bash]
ONBUILD RUN test -z "${NOFRONTEND}"                                &&\
            cd /data && yarn install --frozen-lockfile || :

ONBUILD RUN test -z "${NOFRONTEND}"                                &&\
            cd /data && yarn build --no-progress || :

ONBUILD RUN test -z "${NOFRONTEND}"                                &&\
            mv /data/dist ${TFW_FRONTEND_DIR} && rm -rf /data || :
\end{lstlisting}
Remember that \code{ONBUILD} commands run in the build context of the child image.
What these commands do is they check if the \code{NOFRONTEND} build argument
is present or not, and only deal with the frontend if this argument is not defined.
The \code{|| :} notation in bash basically means ``or true'', which is required
to avoid aborting the build due to the non-zero return code produced
by the \code{test} command if the build arg is defined.

\section{Versioning and Releases}

Currently I use git tags%
\footnote{\href{https://git-scm.com/docs/git-tag}{https://git-scm.com/docs/git-tag}}
to manage releases of the TFW baseimage.
Each new release is a tag digitally signed by me using GnuPG so that
everyone is able to verify that the release is authentic.
The tags are named according to the versioning scheme I have adopted for the project.
I explain this versioning system extensively in a blog post\cite{SemancatVersioning}.

The short version is that we use a solution that is a mix between
semantic versioning%
\footnote{\href{https://semver.org}{https://semver.org}}
and calendar versioning%
\footnote{\href{https://calver.org}{https://calver.org}}.
Our release tags look similar to this one: \code{mainecoon-20180712}
(this was the actual tag of an older release).
The part before the ``\code{-}'' is the major version, which is always named after a breed
of cat, for the additional fun factor, and because we love cats.
The part after that is a timestamp of the day the release was made on.
I only change major versions when I introduce backwards incompatible changes in the
API of the framework, this way developers can trust that releases
with the same majors are compatible with each other in regards to client code.

The \code{master} branches of the frontend-TFW and test-TFW repositories are always
kept compatible with the newest release tag of the baseimage.
This is one of the ways the bootstrap script can operate safely: when cloning the three
repositories, it checks out the newest tag of the baseimage before building it,
ensuring that the newest released version is used for the newly created dev environment.