Compare commits

...
Author SHA1 Message Date
Kristóf Tóth 3b30b333e5 Rename webide -> ide 2018-04-20 17:33:06 +02:00
Kristóf Tóth 9b85c19d40 Make messaging format defined as text to avoid JSON handcuffs 2018-04-19 11:15:19 +02:00
Kristóf Tóth 0b651abb7d Nicen some EventHandler API docstrings 2018-04-19 10:47:20 +02:00
Kristóf Tóth b1592e8ebb Use new f-strings where possible 2018-04-19 09:21:41 +02:00
Kristóf Tóth 3f36826597 Document TFW message format in README 2018-04-18 20:41:16 +02:00
Kristóf Tóth db1b008fc9 Fix EventHandlerBase.event_handler_callback broken logic 2018-04-18 20:16:56 +02:00
Kristóf Tóth 9d93d055b8 Refactor EventHandlerBase to be more general 2018-04-18 20:07:41 +02:00
Kristóf Tóth addd517ba7 Add a huge bunch of docstrings 2018-04-18 19:44:26 +02:00
Kristóf Tóth 690f9bb190 Move TriggerlessEventHandler logic to base class EventHandlerBase 2018-04-18 18:47:51 +02:00
Kristóf Tóth c909317ca0 Make TerminalCommands alias nonexistent commands only 2018-04-18 16:22:01 +02:00
Kristóf Tóth 25d039e84e Include LICENSE in image 2018-04-18 13:38:31 +02:00
Kristóf Tóth 117e7ffc18 Force bash history to be flushed before executing commands 2018-04-18 13:19:51 +02:00
Kristóf Tóth 8e8963b0f8 Simplify TFW baseimage contract 2018-04-17 17:44:38 +02:00
Kristóf Tóth b592a1625a Unify nginx configuration style 2018-04-17 15:14:02 +02:00
Kristóf Tóth 13e90dde41 Fix terminado not being included in nginx config 2018-04-17 15:13:27 +02:00
Kristóf Tóth 63c235bb41 Remove nginx component belonging to test repo 2018-04-17 15:00:43 +02:00
Kristóf Tóth 1061220f82 Add 'where to go next' section to README 2018-04-17 13:57:52 +02:00
Kristóf Tóth ca1278e0b4 Improve documentation 2018-04-17 13:46:32 +02:00
25 changed files with 330 additions and 84 deletions
+7 -11
View File
@@ -39,19 +39,15 @@ ENV PYTHONPATH="/usr/local/lib/" \
TFW_HISTFILE="/home/${AVATAO_USER}/.bash_history" \ TFW_HISTFILE="/home/${AVATAO_USER}/.bash_history" \
PROMPT_COMMAND="history -a" PROMPT_COMMAND="history -a"
RUN echo "shopt -s cmdhist\n" \ COPY bashrc /tmp
"shopt -s histappend\n" \ RUN echo "export HISTFILE=${TFW_HISTFILE}\n" >> /tmp/bashrc &&\
"unset HISTCONTROL\n" \ cat /tmp/bashrc >> /home/${AVATAO_USER}/.bashrc
"export HISTFILE=$TFW_HISTFILE\n" \
"export HISTFILESIZE=1000\n" \
"export HISTSIZE=1000\n" \
'PROMPT_COMMAND="history -a"\n' \
>> /home/${AVATAO_USER}/.bashrc
COPY supervisor/supervisord.conf ${TFW_SUPERVISORD_CONF} COPY supervisor/supervisord.conf ${TFW_SUPERVISORD_CONF}
COPY nginx/nginx.conf ${TFW_NGINX_CONF} COPY nginx/nginx.conf ${TFW_NGINX_CONF}
COPY nginx/default.conf ${TFW_NGINX_DEFAULT} COPY nginx/default.conf ${TFW_NGINX_DEFAULT}
COPY lib ${TFW_LIB_DIR} COPY nginx/components/ ${TFW_NGINX_COMPONENTS}
COPY lib LICENSE ${TFW_LIB_DIR}
RUN for dir in "${TFW_LIB_DIR}" "/etc/nginx" "/etc/supervisor"; do \ RUN for dir in "${TFW_LIB_DIR}" "/etc/nginx" "/etc/supervisor"; do \
chown -R root:root "$dir" && chmod -R 700 "$dir"; \ chown -R root:root "$dir" && chmod -R 700 "$dir"; \
@@ -60,8 +56,8 @@ RUN for dir in "${TFW_LIB_DIR}" "/etc/nginx" "/etc/supervisor"; do \
ONBUILD ARG BUILD_CONTEXT="solvable" ONBUILD ARG BUILD_CONTEXT="solvable"
ONBUILD ARG NOFRONTEND="" ONBUILD ARG NOFRONTEND=""
ONBUILD COPY ${BUILD_CONTEXT}/nginx/components/ ${TFW_NGINX_COMPONENTS} ONBUILD COPY ${BUILD_CONTEXT}/nginx/ ${TFW_NGINX_COMPONENTS}
ONBUILD COPY ${BUILD_CONTEXT}/supervisor/components/ ${TFW_SUPERVISORD_COMPONENTS} ONBUILD COPY ${BUILD_CONTEXT}/supervisor/ ${TFW_SUPERVISORD_COMPONENTS}
ONBUILD RUN for f in "${TFW_NGINX_DEFAULT}" ${TFW_NGINX_COMPONENTS}/*.conf; do \ ONBUILD RUN for f in "${TFW_NGINX_DEFAULT}" ${TFW_NGINX_COMPONENTS}/*.conf; do \
envsubst "$(printenv | cut -d= -f1 | grep TFW_ | sed -e 's/^/$/g')" < $f > $f~ && mv $f~ $f ;\ envsubst "$(printenv | cut -d= -f1 | grep TFW_ | sed -e 's/^/$/g')" < $f > $f~ && mv $f~ $f ;\
+41 -7
View File
@@ -2,7 +2,8 @@
This is the beating heart of TFW the Docker baseimage containing the internals of the framework. This is the beating heart of TFW the Docker baseimage containing the internals of the framework.
All tutorial-framework challenges are child images of this one: their `Dockerfile`s all begin with `FROM avatao/tutorial-framework`. Every tutorial-framework based challenge has a `solvable` Docker image based on this one: their `Dockerfile`s begin with `FROM eu.gcr.io/avatao-challengestore/tutorial-framework`.
Note that TFW is not avaliable on Docker Hub due to legal reasons and is only accessible through local builds (dont't worry, we've got you covered with build scripts in the `test-tutorial-framework` repo).
This document explains the general concepts of TFW and should be the first thing you read before getting started with development. This document explains the general concepts of TFW and should be the first thing you read before getting started with development.
@@ -12,24 +13,25 @@ For more on building and running you should consult the `test-tutorial-framework
The goal of the tutorial-framework is to help content developers in creating interactive tutorials for the Avatao platform. The goal of the tutorial-framework is to help content developers in creating interactive tutorials for the Avatao platform.
To make this possible TFW implements a robust messaging system and provides several pre-written components built upon it, such as a file editor or a terminal (running in your browser). To make this possible TFW implements a robust messaging system and provides several pre-written components built upon it, such as a file editor and a terminal (both running in your browser).
The foundation of the whole framework is the messaging system connecting the frontend with the backend. The foundation of the whole framework is the messaging system connecting the frontend with the backend.
Frontend components use websockets to connect to TFW, to which you can hook several *event handlers* defining how to handle specific messages. Frontend components use websockets to connect to the TFW server, to which you can hook several *event handlers* defining how to handle specific messages.
![TFW architecture](docs/tfw_architecture.png) ![TFW architecture](docs/tfw_architecture.png)
### Event handlers ### Event handlers
Imagine event handlers as callbacks that are executed when TFW receives a specific type of message. For example you could send a message to the framework when the user does something of note. Imagine event handlers as callbacks that are invoked when TFW receives a specific type of message. For instance you could send a message to the framework when the user does something of note.
This allows you to define actions triggered on the backend when the user presses a button on the frontend, moves the cursor to a specific area or anything like that. This allows you to define actions triggered on the backend when the user presses a button on the frontend, moves the cursor to a specific area or anything like that.
Event handlers use ZeroMQ to connect to the framework. They are as loosely-coupled as possible: usually they are running in separate processes and only communicate with TFW through ZMQ. Event handlers use ZeroMQ to connect to the framework. Due to this they are as loosely-coupled as possible: usually they are running in separate processes and only communicate with TFW through ZMQ.
Most of pre-made event handlers are writen in Python3, but you can write event handlers in any language that has ZeroMQ bindings (this means virtually any language). Our pre-made event handlers are written in Python3, but you can write event handlers in any language that has ZeroMQ bindings (this means virtually any language).
This makes the framework really flexible: you can demonstrate the concepts you want to in any language while using the same set of tools provided by TFW. This makes the framework really flexible: you can demonstrate the concepts you want to in any language while using the same set of tools provided by TFW.
Inside Avatao this means that any of the content teams can use the framework with ease.
### FSM ### FSM
@@ -38,7 +40,7 @@ This allows you to track users progressing with the tasks you've defined for the
For instance you could represent whether the user managed to create a malicious user with a state called `user_registered` and subscribe callbacks to events regarding that state (like entering or leaving). For instance you could represent whether the user managed to create a malicious user with a state called `user_registered` and subscribe callbacks to events regarding that state (like entering or leaving).
You could create challenges that can be completed in several different ways: imagine a state called `challenge_complete`, which represents when the challenge is completed. Several series of actions could lead to this state. You could create challenges that can be completed in several different ways: imagine a state called `challenge_complete`, which represents when the challenge is completed. Several series of actions (triggers) could lead to this state.
This enables you to guide your users through the experience you've envisioned with your tutorial. This enables you to guide your users through the experience you've envisioned with your tutorial.
We can provide a whole new level of interactivity in our challenges because we know what the user is doing. We can provide a whole new level of interactivity in our challenges because we know what the user is doing.
@@ -47,3 +49,35 @@ This includes context-dependent hints and the automatic typing of commands to a
### Frontend ### Frontend
Note that our frontend implementation is written in Angular. It is maintained and documented in the `frontend-tutorial-framework` repository. Note that our frontend implementation is written in Angular. It is maintained and documented in the `frontend-tutorial-framework` repository.
### Messaging format
The framework uses JSON messages internally and in exposed APIs as well.
These messages must comply some rules.
Don't worry, we are not too fond of rules around these parts.
The TFW message format:
```text
{
"key: "some identifier used for addressing",
"data":
{
...
JSON object carrying anything, preferably cats
...
},
"trigger": "FSM action"
}
```
- The `key` field is used by TFW for addressing and every message must have one (it can be an empty string though)
- The `data` object can contain anything you might want to send
- The `trigger` key is an optional field that triggers an FSM action with that name from the current state (whatever that might be)
## Where to go next
Most of the components you need have docstrings included (hang on tight, this is work in progress) refer to them for usage info.
To get started you should take a look at the `test-tutorial-framework` repository, which serves as an example project as well.
+20
View File
@@ -0,0 +1,20 @@
_postcmd()
{
AT_PROMPT=1
history -a
}
PROMPT_COMMAND="_postcmd"
_precmd()
{
[ -z "$AT_PROMPT" ] && return
unset AT_PROMPT
history -a
}
trap _precmd DEBUG
shopt -s cmdhist
shopt -s histappend
unset HISTCONTROL
export HISTFILESIZE=1000
export HISTSIZE=1000
Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 40 KiB

+1 -1
View File
@@ -1,6 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft. # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details. # All Rights Reserved. See LICENSE file for details.
from .event_handler_base import TriggerlessEventHandler, TriggeredEventHandler from .event_handler_base import EventHandlerBase, TriggeredEventHandler
from .fsm_base import FSMBase from .fsm_base import FSMBase
from .linear_fsm import LinearFSM from .linear_fsm import LinearFSM
+1 -1
View File
@@ -4,6 +4,6 @@
from .directory_monitoring_event_handler import DirectoryMonitoringEventHandler from .directory_monitoring_event_handler import DirectoryMonitoringEventHandler
from .process_managing_event_handler import ProcessManagingEventHandler from .process_managing_event_handler import ProcessManagingEventHandler
from .terminado_event_handler import TerminadoEventHandler from .terminado_event_handler import TerminadoEventHandler
from .webide_event_handler import WebideEventHandler from .ide_event_handler import IdeEventHandler
from .history_monitor import HistoryMonitor, BashMonitor, GDBMonitor from .history_monitor import HistoryMonitor, BashMonitor, GDBMonitor
from .terminal_commands import TerminalCommands from .terminal_commands import TerminalCommands
+3 -3
View File
@@ -17,7 +17,7 @@ LOG = logging.getLogger(__name__)
class DirectoryMonitor(ObserverMixin): class DirectoryMonitor(ObserverMixin):
def __init__(self, directory): def __init__(self, directory):
ObserverMixin.__init__(self) ObserverMixin.__init__(self)
self.eventhandler = WebideReloadWatchdogEventHandler() self.eventhandler = IdeReloadWatchdogEventHandler()
self.observer.schedule(self.eventhandler, directory, recursive=True) self.observer.schedule(self.eventhandler, directory, recursive=True)
self.pause, self.resume = self.eventhandler.pause, self.eventhandler.resume self.pause, self.resume = self.eventhandler.pause, self.eventhandler.resume
@@ -42,7 +42,7 @@ class DirectoryMonitor(ObserverMixin):
self.directorymonitor.resume() self.directorymonitor.resume()
class WebideReloadWatchdogEventHandler(FileSystemWatchdogEventHandler): class IdeReloadWatchdogEventHandler(FileSystemWatchdogEventHandler):
def __init__(self): def __init__(self):
super().__init__() super().__init__()
self.uplink = ServerUplinkConnector() self.uplink = ServerUplinkConnector()
@@ -63,7 +63,7 @@ class WebideReloadWatchdogEventHandler(FileSystemWatchdogEventHandler):
self.ignore = self.ignore - 1 self.ignore = self.ignore - 1
return return
LOG.debug(event) LOG.debug(event)
self.uplink.send({'key': 'webide', self.uplink.send({'key': 'ide',
'data': {'command': 'reload'}}) 'data': {'command': 'reload'}})
@@ -3,7 +3,7 @@
from os.path import isdir, exists from os.path import isdir, exists
from tfw import TriggerlessEventHandler from tfw import EventHandlerBase
from tfw.config.logs import logging from tfw.config.logs import logging
from tfw.mixins import MonitorManagerMixin from tfw.mixins import MonitorManagerMixin
from .directory_monitor import DirectoryMonitor from .directory_monitor import DirectoryMonitor
@@ -11,7 +11,7 @@ from .directory_monitor import DirectoryMonitor
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
class DirectoryMonitoringEventHandler(TriggerlessEventHandler, MonitorManagerMixin): class DirectoryMonitoringEventHandler(EventHandlerBase, MonitorManagerMixin):
def __init__(self, key, directory): def __init__(self, key, directory):
super().__init__(key) super().__init__(key)
self._directory = directory self._directory = directory
+24
View File
@@ -24,6 +24,17 @@ class CallbackEventHandler(PatternMatchingEventHandler, ABC):
class HistoryMonitor(CallbackMixin, ObserverMixin, ABC): class HistoryMonitor(CallbackMixin, ObserverMixin, ABC):
"""
Abstract class capable of monitoring and parsing a history file such as
bash HISTFILEs. Monitoring means detecting when the file was changed and
notifying subscribers about new content in the file.
This is useful for monitoring CLI sessions.
To specify a custom HistoryMonitor inherit from this class and override the
command pattern property and optionally the sanitize_command method.
See examples below.
"""
def __init__(self, histfile): def __init__(self, histfile):
CallbackMixin.__init__(self) CallbackMixin.__init__(self)
ObserverMixin.__init__(self) ObserverMixin.__init__(self)
@@ -61,6 +72,15 @@ class HistoryMonitor(CallbackMixin, ObserverMixin, ABC):
class BashMonitor(HistoryMonitor): class BashMonitor(HistoryMonitor):
"""
HistoryMonitor for monitoring bash CLI sessions.
This requires the following to be set in bash
(note that this is done automatically by TFW):
PROMPT_COMMAND="history -a"
shopt -s cmdhist
shopt -s histappend
unset HISTCONTROL
"""
@property @property
def command_pattern(self): def command_pattern(self):
return r'.+' return r'.+'
@@ -70,6 +90,10 @@ class BashMonitor(HistoryMonitor):
class GDBMonitor(HistoryMonitor): class GDBMonitor(HistoryMonitor):
"""
HistoryMonitor to monitor GDB sessions.
For this to work "set trace-commands on" must be set in GDB.
"""
@property @property
def command_pattern(self): def command_pattern(self):
return r'(?<=\n)\+(.+)\n' return r'(?<=\n)\+(.+)\n'
@@ -6,7 +6,7 @@ from glob import glob
from fnmatch import fnmatchcase from fnmatch import fnmatchcase
from collections import Iterable from collections import Iterable
from tfw import TriggerlessEventHandler from tfw import EventHandlerBase
from tfw.mixins import MonitorManagerMixin from tfw.mixins import MonitorManagerMixin
from tfw.config.logs import logging from tfw.config.logs import logging
from .directory_monitor import DirectoryMonitor from .directory_monitor import DirectoryMonitor
@@ -40,9 +40,9 @@ class FileManager: # pylint: disable=too-many-instance-attributes
@workdir.setter @workdir.setter
def workdir(self, directory): def workdir(self, directory):
if not exists(directory) or not isdir(directory): if not exists(directory) or not isdir(directory):
raise EnvironmentError('"{}" is not a directory!'.format(directory)) raise EnvironmentError(f'"{directory}" is not a directory!')
if not self._is_in_whitelisted_dir(directory): if not self._is_in_whitelisted_dir(directory):
raise EnvironmentError('Directory "{}" is not in whitelist!'.format(directory)) raise EnvironmentError(f'Directory "{directory}" is not in whitelist!')
self._workdir = directory self._workdir = directory
@property @property
@@ -91,9 +91,25 @@ class FileManager: # pylint: disable=too-many-instance-attributes
return relpath(self._filepath(filename), start=self._workdir) return relpath(self._filepath(filename), start=self._workdir)
class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin): class IdeEventHandler(EventHandlerBase, MonitorManagerMixin):
# pylint: disable=too-many-arguments # pylint: disable=too-many-arguments
"""
Event handler implementing the backend of our browser based IDE.
By default all files in the directory specified in __init__ are displayed
on the fontend. Note that this is a stateful component.
This EventHandler accepts messages that have a data["command"] key specifying
a command to be executed.
The API of each command is documented in their respective handlers.
"""
def __init__(self, key, directory, allowed_directories, selected_file=None, exclude=None): def __init__(self, key, directory, allowed_directories, selected_file=None, exclude=None):
"""
:param key: the key this instance should listen to
:param directory: working directory which the EventHandler should serve files from
:param allowed_directories: list of directories that can be switched to using the selectdir command
:param selected_file: file that is selected by default
:param exclude: list of filenames that should not appear between files (for *.o, *.pyc, etc.)
"""
super().__init__(key) super().__init__(key)
self.filemanager = FileManager(allowed_directories=allowed_directories, working_directory=directory, self.filemanager = FileManager(allowed_directories=allowed_directories, working_directory=directory,
selected_file=selected_file, exclude=exclude) selected_file=selected_file, exclude=exclude)
@@ -106,6 +122,11 @@ class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin):
'exclude': self.exclude} 'exclude': self.exclude}
def read(self, data): def read(self, data):
"""
Read the currently selected file.
:return: message with the contents of the file in data['content']
"""
try: try:
data['content'] = self.filemanager.file_contents data['content'] = self.filemanager.file_contents
except PermissionError: except PermissionError:
@@ -117,6 +138,12 @@ class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin):
return data return data
def write(self, data): def write(self, data):
"""
Overwrites a file with the desired string.
:param data: TFW message data containing keys:
|-string: containing the desired file contents
"""
self.monitor.ignore = self.monitor.ignore + 1 self.monitor.ignore = self.monitor.ignore + 1
try: try:
self.filemanager.file_contents = data['content'] self.filemanager.file_contents = data['content']
@@ -126,6 +153,12 @@ class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin):
return data return data
def select(self, data): def select(self, data):
"""
Selects a file from the current directory.
:param data: TFW message data containing keys:
|-filename: name of file to select relative to the current directory
"""
try: try:
self.filemanager.filename = data['filename'] self.filemanager.filename = data['filename']
except EnvironmentError: except EnvironmentError:
@@ -133,6 +166,14 @@ class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin):
return data return data
def select_dir(self, data): def select_dir(self, data):
"""
Select a new working directory to display files from.
:param data: TFW message data containing keys:
|-directory: absolute path of diretory to select.
must be a path whitelisted in
self.allowed_directories
"""
try: try:
self.filemanager.workdir = data['directory'] self.filemanager.workdir = data['directory']
self.reload_monitor() self.reload_monitor()
@@ -146,6 +187,12 @@ class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin):
return data return data
def exclude(self, data): def exclude(self, data):
"""
Overwrite list of excluded files
:param data: TFW message data containing keys:
|-exclude: list of filename patterns to be excluded, e.g.: ["*.pyc", "*.o"]
"""
try: try:
self.filemanager.exclude = list(data['exclude']) self.filemanager.exclude = list(data['exclude'])
except TypeError: except TypeError:
@@ -153,6 +200,9 @@ class WebideEventHandler(TriggerlessEventHandler, MonitorManagerMixin):
return data return data
def attach_fileinfo(self, data): def attach_fileinfo(self, data):
"""
Basic information included in every response to the frontend.
"""
data['filename'] = self.filemanager.filename data['filename'] = self.filemanager.filename
data['files'] = self.filemanager.files data['files'] = self.filemanager.files
data['directory'] = self.filemanager.workdir data['directory'] = self.filemanager.workdir
@@ -3,7 +3,7 @@
from xmlrpc.client import Fault as SupervisorFault from xmlrpc.client import Fault as SupervisorFault
from tfw import TriggerlessEventHandler from tfw import EventHandlerBase
from tfw.mixins import SupervisorMixin from tfw.mixins import SupervisorMixin
from tfw.config.logs import logging from tfw.config.logs import logging
from .directory_monitor import with_monitor_paused from .directory_monitor import with_monitor_paused
@@ -22,7 +22,19 @@ class ProcessManager(SupervisorMixin):
return self.commands[command](process_name) return self.commands[command](process_name)
class ProcessManagingEventHandler(TriggerlessEventHandler): class ProcessManagingEventHandler(EventHandlerBase):
"""
Event handler that can manage processes managed by supervisor.
This EventHandler accepts messages that have a data["command"] key specifying
a command to be executed.
Every message must contain a data["process_name"] field with the name of the
process to manage. This is the name specified in supervisor config files like so:
[program:someprogram]
Commands available: start, stop, restart, readlog
(the names are as self-documenting as it gets)
"""
def __init__(self, key, dirmonitor=None): def __init__(self, key, dirmonitor=None):
super().__init__(key) super().__init__(key)
self.key = key self.key = key
+29 -2
View File
@@ -1,7 +1,7 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft. # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details. # All Rights Reserved. See LICENSE file for details.
from tfw import TriggerlessEventHandler from tfw import EventHandlerBase
from tfw.config import TFWENV from tfw.config import TFWENV
from tfw.config.logs import logging from tfw.config.logs import logging
from tao.config import TAOENV from tao.config import TAOENV
@@ -10,8 +10,21 @@ from .terminado_mini_server import TerminadoMiniServer
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
class TerminadoEventHandler(TriggerlessEventHandler): class TerminadoEventHandler(EventHandlerBase):
"""
Event handler responsible for managing terminal sessions for frontend xterm
sessions to connect to. You need to instanciate this in order for frontend
terminals to work.
This EventHandler accepts messages that have a data["command"] key specifying
a command to be executed.
The API of each command is documented in their respective handlers.
"""
def __init__(self, key, monitor): def __init__(self, key, monitor):
"""
:param key: key this EventHandler listens to
:param monitor: tfw.components.HistoryMonitor instance to read command history from
"""
super().__init__(key) super().__init__(key)
self.working_directory = TFWENV.TERMINADO_DIR self.working_directory = TFWENV.TERMINADO_DIR
self._historymonitor = monitor self._historymonitor = monitor
@@ -37,9 +50,23 @@ class TerminadoEventHandler(TriggerlessEventHandler):
LOG.error('IGNORING MESSAGE: Invalid message received: %s', message) LOG.error('IGNORING MESSAGE: Invalid message received: %s', message)
def write(self, data): def write(self, data):
"""
Writes a string to the terminal session (on the pty level).
Useful for pre-typing and executing commands for the user.
:param data: TFW message data containing keys:
|-shellcmd: command to be written to the pty
"""
self.terminado_server.pty.write(data['shellcmd']) self.terminado_server.pty.write(data['shellcmd'])
def read(self, data): def read(self, data):
"""
Reads the history of commands executed.
:param data: TFW message data containing keys:
|-count: the number of history elements to return
:return: message with list of commands in data['history']
"""
data['count'] = int(data.get('count', 1)) data['count'] = int(data.get('count', 1))
if self.historymonitor: if self.historymonitor:
data['history'] = self.historymonitor.history[-data['count']:] data['history'] = self.historymonitor.history[-data['count']:]
+19 -1
View File
@@ -10,6 +10,24 @@ LOG = logging.getLogger(__name__)
class TerminalCommands(ABC): class TerminalCommands(ABC):
"""
A class you can use to define hooks for terminal commands. This means that you can
have python code executed when the user enters a specific command to the terminal on
our frontend.
To receive events you need to subscribe TerminalCommand.callback to a HistoryMonitor
instance.
Inherit from this class and define methods which start with "command_". When the user
executes the command specified after the underscore, your method will be invoked. All
such commands must expect the parameter *args which will contain the arguments of the
command.
For example to define a method that runs when someone starts vim in the terminal
you have to define a method like: "def command_vim(self, *args)"
You can also use this class to create new commands similarly.
"""
def __init__(self, bashrc=None): def __init__(self, bashrc=None):
self._command_method_regex = r'^command_(.+)$' self._command_method_regex = r'^command_(.+)$'
self.command_implemetations = self._build_command_to_implementation_dict() self.command_implemetations = self._build_command_to_implementation_dict()
@@ -22,7 +40,7 @@ class TerminalCommands(ABC):
def _setup_bashrc_aliases(self, bashrc): def _setup_bashrc_aliases(self, bashrc):
with open(bashrc, 'a') as ofile: with open(bashrc, 'a') as ofile:
alias_template = 'alias {0}="{0} > /dev/null 2>&1"\n' alias_template = 'type {0} &> /dev/null || alias {0}="{0} &> /dev/null"\n'
for command in self.command_implemetations.keys(): for command in self.command_implemetations.keys():
ofile.write(alias_template.format(command)) ofile.write(alias_template.format(command))
+58 -36
View File
@@ -8,75 +8,97 @@ from tfw.networking.event_handlers import ServerConnector
class EventHandlerBase(ABC): class EventHandlerBase(ABC):
"""
Abstract base class for all Python based EventHandlers. Useful implementation template
for other languages.
Derived classes must implement the handle_event() method
"""
def __init__(self, key): def __init__(self, key):
self.server_connector = ServerConnector() self.server_connector = ServerConnector()
self.key = key self.key = key
self.subscriptions = set() self.subscribe(self.key, 'reset')
self.subscribe(self.key)
self.subscribe('reset')
self.server_connector.register_callback(self.event_handler_callback) self.server_connector.register_callback(self.event_handler_callback)
def event_handler_callback(self, msg_parts): def event_handler_callback(self, msg_parts):
"""
Callback that is invoked when receiving a message.
Dispatches messages to handler methods and sends
a response back in case the handler returned something.
This is subscribed in __init__().
"""
message = deserialize_tfw_msg(*msg_parts) message = deserialize_tfw_msg(*msg_parts)
response = self.dispatch_handling(message) response = self.dispatch_handling(message)
response['key'] = message['key'] if response:
if response is None: response['key'] = message['key']
return self.server_connector.send(response)
self.server_connector.send(response)
@abstractmethod
def dispatch_handling(self, message): def dispatch_handling(self, message):
raise NotImplementedError """
Used to dispatch messages to their specific handlers.
def _dispatch_handling(self, message): """
# pylint: disable=no-else-return
if message['key'] != 'reset': if message['key'] != 'reset':
return self.handle_event(message) return self.handle_event(message)
else: return self.handle_reset(message)
return self.handle_reset(message)
@abstractmethod @abstractmethod
def handle_event(self, message): def handle_event(self, message):
"""
Abstract method that implements the handling of messages.
:param message: the message received
"""
raise NotImplementedError raise NotImplementedError
def handle_reset(self, message): def handle_reset(self, message):
# pylint: disable=unused-argument,no-self-use # pylint: disable=unused-argument,no-self-use
"""
Usually 'reset' events receive some sort of special treatment.
:param message: the message received
"""
return None return None
def cleanup(self): def subscribe(self, *keys):
pass """
Subscribe this EventHandler to receive events for given keys.
def subscribe(self, key): Note that you can subscribe to the same key several times in which
if key not in self.subscriptions: case you will need to unsubscribe multiple times in order to stop
self.subscriptions.add(key) receiving events.
:param keys: list of keys to subscribe to
"""
for key in keys:
self.server_connector.subscribe(key) self.server_connector.subscribe(key)
def unsubscribe(self, key): def unsubscribe(self, *keys):
try: """
self.subscriptions.remove(key) Unsubscribe this eventhandler from the given keys.
:param keys: list of keys to unsubscribe from
"""
for key in keys:
self.server_connector.unsubscribe(key) self.server_connector.unsubscribe(key)
except KeyError:
pass
def unsubscribe_all(self): def cleanup(self):
for sub in self.subscriptions: """
self.server_connector.unsubscribe(key=sub) Perform cleanup actions such as releasing database
self.subscriptions.clear() connections and stuff like that.
"""
pass
class TriggerlessEventHandler(EventHandlerBase, ABC):
# pylint: disable=abstract-method
def dispatch_handling(self, message):
return self._dispatch_handling(message)
class TriggeredEventHandler(EventHandlerBase, ABC): class TriggeredEventHandler(EventHandlerBase, ABC):
# pylint: disable=abstract-method # pylint: disable=abstract-method
"""
Abstract base class for EventHandlers which are only triggered in case
TFWServer has successfully triggered an FSM step defined in __init__.
"""
def __init__(self, key, trigger): def __init__(self, key, trigger):
super().__init__(key) super().__init__(key)
self.trigger = trigger self.trigger = trigger
def dispatch_handling(self, message): def dispatch_handling(self, message):
if message.get('trigger') == self.trigger: if message.get('trigger') == self.trigger:
return self._dispatch_handling(message) return super().dispatch_handling(message)
return None return None
+7
View File
@@ -9,6 +9,13 @@ from tfw.mixins import CallbackMixin
class FSMBase(CallbackMixin): class FSMBase(CallbackMixin):
"""
A general FSM base class you can inherit from to track user progress.
See linear_fsm.py for an example use-case.
TFW the transitions library for state machines, please refer to their
documentation for more information on creating your own machines:
https://github.com/pytransitions/transitions
"""
states, transitions = [], [] states, transitions = [], []
def __init__(self, initial: str = None, accepted_states: List[str] = None): def __init__(self, initial: str = None, accepted_states: List[str] = None):
+7 -1
View File
@@ -5,8 +5,14 @@ from .fsm_base import FSMBase
class LinearFSM(FSMBase): class LinearFSM(FSMBase):
"""
This is a state machine for challenges with linear progression, consisting of
a number of steps specified in the constructor. It automatically sets up a single
action between states as such:
0 ==step_1==> 1 ==step_2==> 2 ==step_3==> 3 ... and so on
"""
def __init__(self, number_of_steps): def __init__(self, number_of_steps):
self.states = list(map(str, range(number_of_steps))) self.states = list(map(str, range(number_of_steps)))
self.transitions = [{'trigger': 'step_{}'.format(int(index)+1), 'source': index, 'dest': str(int(index)+1)} self.transitions = [{'trigger': f'step_{int(index)+1}', 'source': index, 'dest': str(int(index)+1)}
for index in self.states[:-1]] for index in self.states[:-1]]
super(LinearFSM, self).__init__() super(LinearFSM, self).__init__()
+6
View File
@@ -9,6 +9,12 @@ class CallbackMixin:
self._callbacks = [] self._callbacks = []
def subscribe_callback(self, callback, *args, **kwargs): def subscribe_callback(self, callback, *args, **kwargs):
"""
Subscribe a callable to invoke once an event is triggered.
:param callback: callable to be executed on events
:param *args: arguments passed to callable
:param **kwargs: kwargs passed to callable
"""
fun = partial(callback, *args, **kwargs) fun = partial(callback, *args, **kwargs)
self._callbacks.append(fun) self._callbacks.append(fun)
+1 -1
View File
@@ -12,7 +12,7 @@ class ControllerConnector(ZMQConnectorBase):
def __init__(self, zmq_context=None): def __init__(self, zmq_context=None):
super(ControllerConnector, self).__init__(zmq_context) super(ControllerConnector, self).__init__(zmq_context)
self._zmq_rep_socket = self._zmq_context.socket(zmq.REP) self._zmq_rep_socket = self._zmq_context.socket(zmq.REP)
self._zmq_rep_socket.connect('tcp://localhost:{}'.format(TFWENV.CONTROLLER_PORT)) self._zmq_rep_socket.connect(f'tcp://localhost:{TFWENV.CONTROLLER_PORT}')
self._zmq_rep_stream = ZMQStream(self._zmq_rep_socket) self._zmq_rep_stream = ZMQStream(self._zmq_rep_socket)
self.register_callback = self._zmq_rep_stream.on_recv_stream self.register_callback = self._zmq_rep_stream.on_recv_stream
@@ -15,7 +15,7 @@ class ServerDownlinkConnector(ZMQConnectorBase):
def __init__(self, zmq_context=None): def __init__(self, zmq_context=None):
super(ServerDownlinkConnector, self).__init__(zmq_context) super(ServerDownlinkConnector, self).__init__(zmq_context)
self._zmq_sub_socket = self._zmq_context.socket(zmq.SUB) self._zmq_sub_socket = self._zmq_context.socket(zmq.SUB)
self._zmq_sub_socket.connect('tcp://localhost:{}'.format(TFWENV.PUBLISHER_PORT)) self._zmq_sub_socket.connect(f'tcp://localhost:{TFWENV.PUBLISHER_PORT}')
self._zmq_sub_stream = ZMQStream(self._zmq_sub_socket) self._zmq_sub_stream = ZMQStream(self._zmq_sub_socket)
self.subscribe = partial(self._zmq_sub_socket.setsockopt_string, zmq.SUBSCRIBE) self.subscribe = partial(self._zmq_sub_socket.setsockopt_string, zmq.SUBSCRIBE)
@@ -24,18 +24,30 @@ class ServerDownlinkConnector(ZMQConnectorBase):
class ServerUplinkConnector(ZMQConnectorBase): class ServerUplinkConnector(ZMQConnectorBase):
"""
Class capable of sending messages to the TFW server and event handlers.
"""
def __init__(self, zmq_context=None): def __init__(self, zmq_context=None):
super(ServerUplinkConnector, self).__init__(zmq_context) super(ServerUplinkConnector, self).__init__(zmq_context)
self._zmq_push_socket = self._zmq_context.socket(zmq.PUSH) self._zmq_push_socket = self._zmq_context.socket(zmq.PUSH)
self._zmq_push_socket.connect('tcp://localhost:{}'.format(TFWENV.RECEIVER_PORT)) self._zmq_push_socket.connect(f'tcp://localhost:{TFWENV.RECEIVER_PORT}')
def send_to_eventhandler(self, message): def send_to_eventhandler(self, message):
"""
Send a message to an event handler.
:param message: JSON message you want to send
:param message['key']: key of event handler you want to address
"""
nested_message = {'key': message['key'], 'data': message.pop('data')} nested_message = {'key': message['key'], 'data': message.pop('data')}
message['key'] = 'mirror' message['key'] = 'mirror'
message['data'] = nested_message message['data'] = nested_message
self.send(message) self.send(message)
def send(self, message): def send(self, message):
"""
Send a message to the TFW server
:param message: JSON message you want to send
"""
self._zmq_push_socket.send_multipart(serialize_tfw_msg(message)) self._zmq_push_socket.send_multipart(serialize_tfw_msg(message))
+9
View File
@@ -7,11 +7,20 @@ from tfw.networking.event_handlers import ServerUplinkConnector
class MessageSender: class MessageSender:
"""
Provides a mechanism to send messages to our frontend messaging component which
displays messages with the key "message".
"""
def __init__(self, custom_key: str = None): def __init__(self, custom_key: str = None):
self.server_connector = ServerUplinkConnector() self.server_connector = ServerUplinkConnector()
self.key = custom_key or 'message' self.key = custom_key or 'message'
def send(self, originator, message): def send(self, originator, message):
"""
Sends a message to the key specified in __init__.
:param originator: name of sender to be displayed on the frontend
:param message: message to send
"""
data = { data = {
'originator': originator, 'originator': originator,
'timestamp': datetime.now().isoformat(), 'timestamp': datetime.now().isoformat(),
@@ -16,7 +16,7 @@ class EventHandlerDownlinkConnector(ZMQConnectorBase):
super(EventHandlerDownlinkConnector, self).__init__(zmq_context) super(EventHandlerDownlinkConnector, self).__init__(zmq_context)
self._zmq_pull_socket = self._zmq_context.socket(zmq.PULL) self._zmq_pull_socket = self._zmq_context.socket(zmq.PULL)
self._zmq_pull_stream = ZMQStream(self._zmq_pull_socket) self._zmq_pull_stream = ZMQStream(self._zmq_pull_socket)
address = 'tcp://*:{}'.format(TFWENV.RECEIVER_PORT) address = f'tcp://*:{TFWENV.RECEIVER_PORT}'
self._zmq_pull_socket.bind(address) self._zmq_pull_socket.bind(address)
LOG.debug('Pull socket bound to %s', address) LOG.debug('Pull socket bound to %s', address)
@@ -25,7 +25,7 @@ class EventHandlerUplinkConnector(ZMQConnectorBase):
def __init__(self, zmq_context=None): def __init__(self, zmq_context=None):
super(EventHandlerUplinkConnector, self).__init__(zmq_context) super(EventHandlerUplinkConnector, self).__init__(zmq_context)
self._zmq_pub_socket = self._zmq_context.socket(zmq.PUB) self._zmq_pub_socket = self._zmq_context.socket(zmq.PUB)
address = 'tcp://*:{}'.format(TFWENV.PUBLISHER_PORT) address = f'tcp://*:{TFWENV.PUBLISHER_PORT}'
self._zmq_pub_socket.bind(address) self._zmq_pub_socket.bind(address)
LOG.debug('Pub socket bound to %s', address) LOG.debug('Pub socket bound to %s', address)
+8
View File
@@ -15,7 +15,15 @@ LOG = logging.getLogger(__name__)
class TFWServer: class TFWServer:
"""
This class handles the proxying of messages between the frontend and event handers.
It proxies messages from the "/ws" route to all event handlers subscribed to a ZMQ
SUB socket. It also manages an FSM you can define as a constructor argument.
"""
def __init__(self, fsm_type): def __init__(self, fsm_type):
"""
:param fsm_type: the type of FSM you want TFW to use
"""
self._fsm = fsm_type() self._fsm = fsm_type()
self._fsm_updater = FSMUpdater(self._fsm) self._fsm_updater = FSMUpdater(self._fsm)
self._fsm_manager = FSMManager(self._fsm) self._fsm_manager = FSMManager(self._fsm)
-3
View File
@@ -1,3 +0,0 @@
location = /login {
proxy_pass http://127.0.0.1:${TFW_LOGIN_APP_PORT};
}
+1 -1
View File
@@ -3,4 +3,4 @@ location = /terminal {
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade; proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade"; proxy_set_header Connection "upgrade";
} }
+2 -4
View File
@@ -1,13 +1,11 @@
worker_processes auto; worker_processes auto;
pid /tmp/nginx.pid; pid /tmp/nginx.pid;
events events {
{
worker_connections 1024; worker_connections 1024;
} }
http http {
{
sendfile on; sendfile on;
tcp_nopush on; tcp_nopush on;
tcp_nodelay on; tcp_nodelay on;