Compare commits

...
Author SHA1 Message Date
Bokros Bálint c05f9f6a28 Merge pull request #26 from avatao-content/niceimports
Niceimports
2018-04-06 16:24:37 +02:00
Kristóf Tóth 11743e830a Expose ServerUplinkConnector with a name understandable without TFW knowledge 2018-04-06 16:22:21 +02:00
Kristóf Tóth 0464acbdc4 Make error logs more explicit 2018-04-06 16:09:05 +02:00
Kristóf Tóth 64bd2f1ba0 Refactor handling of shared EventHandlerConnector ownership to nicen imports 2018-04-06 16:04:25 +02:00
Kristóf Tóth 1f2e1f0489 Refactor whole importing hell into a nice and clean API 2018-04-06 15:31:16 +02:00
Bokros Bálint b942c1cf53 Merge pull request #25 from avatao-content/usersep
Fix webide vulnerabilities
2018-04-05 17:22:04 +02:00
Kristóf Tóth bd84e4fe06 Improve code formatting in webide 2018-04-05 17:19:56 +02:00
Kristóf Tóth bc340e2e19 Enforce webide whitelisting 2018-04-05 17:16:41 +02:00
Kristóf Tóth b73b7307bd Improve webide whitelisting by enforcing stricter rules (fix symlink attack) 2018-04-05 17:01:50 +02:00
Kristóf Tóth b74ff39438 Implement directory whitelisting in webide 2018-04-05 14:43:39 +02:00
Kristóf Tóth 35421649c9 Extract spawning bash as user to a variable 2018-04-05 14:43:07 +02:00
Kristóf Tóth b54c91848b Remove tfw server init from baseimage (part of child contract now) 2018-04-04 17:48:49 +02:00
Kristóf Tóth 19f819c142 Implement TFW & challenge solver user separation 2018-04-04 17:43:18 +02:00
Kristóf Tóth 68fc4ca050 Fix nginx on readonly filesystems 2018-04-04 14:18:37 +02:00
Kristóf Tóth ade0936c6f Provide high-level documentation of event handlers and FSM 2018-04-03 17:41:19 +02:00
Kristóf Tóth d072f9ee9c Extend documentation with framework basics 2018-04-03 16:20:12 +02:00
Kristóf Tóth eaa0eee4ca Add LICENSE file and include copyright notice in source files 2018-04-03 14:49:14 +02:00
Kristóf Tóth b8bcda134c Start rewriting README.md 2018-03-30 22:37:36 +02:00
40 changed files with 256 additions and 67 deletions
+6 -2
View File
@@ -53,16 +53,20 @@ COPY nginx/nginx.conf ${TFW_NGINX_CONF}
COPY nginx/default.conf ${TFW_NGINX_DEFAULT} COPY nginx/default.conf ${TFW_NGINX_DEFAULT}
COPY lib ${TFW_LIB_DIR} COPY lib ${TFW_LIB_DIR}
RUN for dir in "${TFW_LIB_DIR}" "/etc/nginx" "/etc/supervisor"; do \
chown -R root:root "$dir" && chmod -R 700 "$dir"; \
done
ONBUILD ARG BUILD_CONTEXT="." ONBUILD ARG BUILD_CONTEXT="."
ONBUILD ARG NOFRONTEND="" ONBUILD ARG NOFRONTEND=""
ONBUILD COPY ${BUILD_CONTEXT}/nginx/components/ ${TFW_NGINX_COMPONENTS} ONBUILD COPY ${BUILD_CONTEXT}/nginx/components/ ${TFW_NGINX_COMPONENTS}
ONBUILD COPY ${BUILD_CONTEXT}/supervisor/components/ ${TFW_SUPERVISORD_COMPONENTS} ONBUILD COPY ${BUILD_CONTEXT}/supervisor/components/ ${TFW_SUPERVISORD_COMPONENTS}
ONBUILD RUN chown -R ${AVATAO_USER} /var/log/nginx /var/lib/nginx &&\ ONBUILD RUN for f in "${TFW_NGINX_DEFAULT}" ${TFW_NGINX_COMPONENTS}/*.conf; do \
for f in "${TFW_NGINX_DEFAULT}" ${TFW_NGINX_COMPONENTS}/*.conf; do \
envsubst "$(printenv | cut -d= -f1 | grep TFW_ | sed -e 's/^/$/g')" < $f > $f~ && mv $f~ $f ;\ envsubst "$(printenv | cut -d= -f1 | grep TFW_ | sed -e 's/^/$/g')" < $f > $f~ && mv $f~ $f ;\
done done
ONBUILD VOLUME ["/etc/nginx", "/var/lib/nginx", "/var/log/nginx"]
ONBUILD COPY ${BUILD_CONTEXT}/frontend /data/ ONBUILD COPY ${BUILD_CONTEXT}/frontend /data/
ONBUILD RUN test -z "${NOFRONTEND}" && cd /data && yarn install --frozen-lockfile || : ONBUILD RUN test -z "${NOFRONTEND}" && cd /data && yarn install --frozen-lockfile || :
+12
View File
@@ -0,0 +1,12 @@
AVATAO CONFIDENTIAL
Copyright (C) 2018 Avatao.com Innovative Learning Kft.
All Rights Reserved.
All source code, configuration files and documentation contained herein
is, and remains the exclusive property of Avatao.com Innovative Learning Kft.
The intellectual and technical concepts contained herein are proprietary
to Avatao.com Innovative Learning Kft. and are protected by trade secret
or copyright law. Dissemination of this information or reproduction of
this material is strictly forbidden unless prior written permission is
obtained from Avatao.com Innovative Learning Kft.
+42 -9
View File
@@ -1,16 +1,49 @@
Tutorial Framework # baseimage-tutorial-framework
==================
## Building and running with magic starter This is the beating heart of TFW the Docker baseimage containing the internals of the framework.
Execute the `tfw_magic_start.sh` script and it will automagically run a backend instance in Docker and serve the Angular frontend locally. All tutorial-framework challenges are child images of this one: their `Dockerfile`s all begin with `FROM avatao/tutorial-framework`.
This requires the `test-tutorial-framework` (Docker image `FROM baseimage-tutorial-framework`) and `frontend-tutorial-framework` repositories to have a common parent folder. This document explains the general concepts of TFW and should be the first thing you read before getting started with development.
## Building with Docker For more on building and running you should consult the `test-tutorial-framework` repo.
Simply issue `docker build -t baseimage-tutorial-framework .` in the project root. ## The framework
## Creating child images The goal of the tutorial-framework is to help content developers in creating interactive tutorials for the Avatao platform.
Documentation in progress... To make this possible TFW implements a robust messaging system and provides several pre-written components built upon it, such as a file editor or a terminal (running in your browser).
The foundation of the whole framework is the messaging system connecting the frontend with the backend.
Frontend components use websockets to connect to TFW, to which you can hook several *event handlers* defining how to handle specific messages.
![TFW architecture](docs/tfw_architecture.png)
### Event handlers
Imagine event handlers as callbacks that are executed when TFW receives a specific type of message. For example you could send a message to the framework when the user does something of note.
This allows you to define actions triggered on the backend when the user presses a button on the frontend, moves the cursor to a specific area or anything like that.
Event handlers use ZeroMQ to connect to the framework. They are as loosely-coupled as possible: usually they are running in separate processes and only communicate with TFW through ZMQ.
Most of pre-made event handlers are writen in Python3, but you can write event handlers in any language that has ZeroMQ bindings (this means virtually any language).
This makes the framework really flexible: you can demonstrate the concepts you want to in any language while using the same set of tools provided by TFW.
### FSM
Another unique feature of the framework is the FSM finite state machine representing the state of your challenge.
This allows you to track users progressing with the tasks you've defined for them to complete.
For instance you could represent whether the user managed to create a malicious user with a state called `user_registered` and subscribe callbacks to events regarding that state (like entering or leaving).
You could create challenges that can be completed in several different ways: imagine a state called `challenge_complete`, which represents when the challenge is completed. Several series of actions could lead to this state.
This enables you to guide your users through the experience you've envisioned with your tutorial.
We can provide a whole new level of interactivity in our challenges because we know what the user is doing.
This includes context-dependent hints and the automatic typing of commands to a terminal.
### Frontend
Note that our frontend implementation is written in Angular. It is maintained and documented in the `frontend-tutorial-framework` repository.
Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

+3
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from collections import namedtuple from collections import namedtuple
from os import environ from os import environ
+3
View File
@@ -0,0 +1,3 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
+4 -1
View File
@@ -1 +1,4 @@
from .envvars import * # pylint: disable=wildcard-import # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .envvars import TAOENV
+3
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from envvars import prefixed_envvars_to_namedtuple from envvars import prefixed_envvars_to_namedtuple
TAOENV = prefixed_envvars_to_namedtuple('AVATAO_', 'taoenvtuple') TAOENV = prefixed_envvars_to_namedtuple('AVATAO_', 'taoenvtuple')
+5
View File
@@ -0,0 +1,5 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .event_handler_base import TriggerlessEventHandler, TriggeredEventHandler
from .fsm_base import FSMBase
+8
View File
@@ -0,0 +1,8 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .directory_monitoring_event_handler import DirectoryMonitoringEventHandler
from .process_managing_event_handler import ProcessManagingEventHandler
from .terminado_event_handler import TerminadoEventHandler
from .source_code_event_handler import SourceCodeEventHandler
from .history_monitor import HistoryMonitor, BashMonitor, GDBMonitor
@@ -1 +1,4 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .rate_limiter import RateLimiter from .rate_limiter import RateLimiter
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from functools import wraps from functools import wraps
from time import time, sleep from time import time, sleep
+4 -1
View File
@@ -1,9 +1,12 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from functools import wraps from functools import wraps
from watchdog.observers import Observer from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler as FileSystemWatchdogEventHandler from watchdog.events import FileSystemEventHandler as FileSystemWatchdogEventHandler
from tfw.networking.event_handlers.server_connector import ServerUplinkConnector from tfw.networking.event_handlers import ServerUplinkConnector
from tfw.components.decorators import RateLimiter from tfw.components.decorators import RateLimiter
from tfw.config.logs import logging from tfw.config.logs import logging
@@ -1,8 +1,11 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from os.path import isdir, exists from os.path import isdir, exists
from tfw.components.directory_monitor import DirectoryMonitor from tfw import TriggerlessEventHandler
from tfw.event_handler_base import TriggerlessEventHandler
from tfw.config.logs import logging from tfw.config.logs import logging
from .directory_monitor import DirectoryMonitor
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
+5 -2
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from os.path import dirname from os.path import dirname
from re import findall from re import findall
from re import compile as compileregex from re import compile as compileregex
@@ -6,8 +9,8 @@ from abc import ABC, abstractmethod
from watchdog.observers import Observer from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler from watchdog.events import PatternMatchingEventHandler
from tfw.components.mixins.callback_mixin import CallbackMixin from tfw.components.mixins import CallbackMixin
from tfw.components.decorators.rate_limiter import RateLimiter from tfw.components.decorators import RateLimiter
class CallbackEventHandler(PatternMatchingEventHandler, ABC): class CallbackEventHandler(PatternMatchingEventHandler, ABC):
+4
View File
@@ -1 +1,5 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .supervisor_mixin import SupervisorMixin from .supervisor_mixin import SupervisorMixin
from .callback_mixin import CallbackMixin
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from functools import partial from functools import partial
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import xmlrpc.client import xmlrpc.client
from xmlrpc.client import Fault as SupervisorFault from xmlrpc.client import Fault as SupervisorFault
from contextlib import suppress from contextlib import suppress
@@ -1,9 +1,12 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from xmlrpc.client import Fault as SupervisorFault from xmlrpc.client import Fault as SupervisorFault
from tfw.event_handler_base import TriggerlessEventHandler from tfw import TriggerlessEventHandler
from tfw.components.mixins import SupervisorMixin from tfw.components.mixins import SupervisorMixin
from tfw.components.directory_monitor import with_monitor_paused
from tfw.config.logs import logging from tfw.config.logs import logging
from .directory_monitor import with_monitor_paused
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
+31 -11
View File
@@ -1,18 +1,22 @@
from os.path import isfile, join, relpath, exists, isdir # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from os.path import isfile, join, relpath, exists, isdir, realpath
from glob import glob from glob import glob
from fnmatch import fnmatchcase from fnmatch import fnmatchcase
from collections import Iterable from collections import Iterable
from tfw.event_handler_base import TriggerlessEventHandler from tfw import TriggerlessEventHandler
from tfw.components.directory_monitor import DirectoryMonitor
from tfw.config.logs import logging from tfw.config.logs import logging
from .directory_monitor import DirectoryMonitor
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
class FileManager: class FileManager: # pylint: disable=too-many-instance-attributes
def __init__(self, working_directory, selected_file=None, exclude=None): def __init__(self, working_directory, allowed_directories, selected_file=None, exclude=None):
self._exclude, self.exclude = None, exclude self._exclude, self.exclude = None, exclude
self._allowed_directories, self.allowed_directories = None, allowed_directories
self._workdir, self.workdir = None, working_directory self._workdir, self.workdir = None, working_directory
self._filename, self.filename = None, selected_file or self.files[0] self._filename, self.filename = None, selected_file or self.files[0]
@@ -36,8 +40,18 @@ class FileManager:
def workdir(self, directory): def workdir(self, directory):
if not exists(directory) or not isdir(directory): if not exists(directory) or not isdir(directory):
raise EnvironmentError('"{}" is not a directory!'.format(directory)) raise EnvironmentError('"{}" is not a directory!'.format(directory))
if not self._is_whitelisted(directory):
raise EnvironmentError('Directory "{}" is not in whitelist!'.format(directory))
self._workdir = directory self._workdir = directory
@property
def allowed_directories(self):
return self._allowed_directories
@allowed_directories.setter
def allowed_directories(self, directories):
self._allowed_directories = directories
@property @property
def filename(self): def filename(self):
return self._filename return self._filename
@@ -51,8 +65,9 @@ class FileManager:
@property @property
def files(self): def files(self):
return [self._relpath(file) for file in glob(join(self._workdir, '**/*'), recursive=True) return [self._relpath(file) for file in glob(join(self._workdir, '**/*'), recursive=True)
if isfile(file) and if isfile(file)
not any(fnmatchcase(file, blacklisted) for blacklisted in self.exclude)] and self._is_whitelisted(file)
and not any(fnmatchcase(file, blacklisted) for blacklisted in self.exclude)]
@property @property
def file_contents(self): def file_contents(self):
@@ -64,6 +79,9 @@ class FileManager:
with open(self._filepath(self.filename), 'w', errors='surrogateescape') as ofile: with open(self._filepath(self.filename), 'w', errors='surrogateescape') as ofile:
ofile.write(value) ofile.write(value)
def _is_whitelisted(self, file):
return any(realpath(file).startswith(allowed_dir) for allowed_dir in self.allowed_directories)
def _filepath(self, filename): def _filepath(self, filename):
return join(self._workdir, filename) return join(self._workdir, filename)
@@ -72,9 +90,11 @@ class FileManager:
class SourceCodeEventHandler(TriggerlessEventHandler): class SourceCodeEventHandler(TriggerlessEventHandler):
def __init__(self, key, directory, selected_file=None, exclude=None): # pylint: disable=too-many-arguments
def __init__(self, key, directory, allowed_directories, selected_file=None, exclude=None):
super().__init__(key) super().__init__(key)
self.filemanager = FileManager(directory, selected_file=selected_file, exclude=exclude) self.filemanager = FileManager(allowed_directories=allowed_directories, working_directory=directory,
selected_file=selected_file, exclude=exclude)
self.commands = {'read': self.read, self.commands = {'read': self.read,
'write': self.write, 'write': self.write,
@@ -134,8 +154,8 @@ class SourceCodeEventHandler(TriggerlessEventHandler):
self.read(data) self.read(data)
except IndexError: except IndexError:
data['content'] = 'No files in this directory :(' data['content'] = 'No files in this directory :('
except EnvironmentError: except EnvironmentError as err:
LOG.error('Failed to select directory "%s"', data['directory']) LOG.error('Failed to select directory "%s". Reason: %s', data['directory'], str(err))
return data return data
def exclude(self, data): def exclude(self, data):
@@ -1,7 +1,11 @@
from tfw.components.terminado_mini_server import TerminadoMiniServer # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
from tfw.event_handler_base import TriggerlessEventHandler # All Rights Reserved. See LICENSE file for details.
from tfw import TriggerlessEventHandler
from tfw.config import TFWENV from tfw.config import TFWENV
from tfw.config.logs import logging from tfw.config.logs import logging
from tao.config import TAOENV
from .terminado_mini_server import TerminadoMiniServer
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
@@ -11,7 +15,8 @@ class TerminadoEventHandler(TriggerlessEventHandler):
super().__init__(key) super().__init__(key)
self.working_directory = TFWENV.TERMINADO_DIR self.working_directory = TFWENV.TERMINADO_DIR
self._historymonitor = monitor self._historymonitor = monitor
self.terminado_server = TerminadoMiniServer('/terminal', TFWENV.TERMINADO_PORT, TFWENV.TERMINADO_WD, ['bash']) bash_as_user_cmd = ['sudo', '-u', TAOENV.USER, 'bash']
self.terminado_server = TerminadoMiniServer('/terminal', TFWENV.TERMINADO_PORT, TFWENV.TERMINADO_WD, bash_as_user_cmd)
self.commands = {'write': self.write, self.commands = {'write': self.write,
'read': self.read} 'read': self.read}
if self._historymonitor: if self._historymonitor:
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from tornado.ioloop import IOLoop from tornado.ioloop import IOLoop
from tornado.web import Application from tornado.web import Application
from terminado import TermSocket, SingleTermManager from terminado import TermSocket, SingleTermManager
+4 -1
View File
@@ -1 +1,4 @@
from .envvars import * # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .envvars import TFWENV
+3
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from envvars import prefixed_envvars_to_namedtuple from envvars import prefixed_envvars_to_namedtuple
TFWENV = prefixed_envvars_to_namedtuple('TFW_', 'tfwenvtuple') TFWENV = prefixed_envvars_to_namedtuple('TFW_', 'tfwenvtuple')
+3
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import logging import logging
logging.basicConfig(level=logging.DEBUG) logging.basicConfig(level=logging.DEBUG)
+5 -2
View File
@@ -1,7 +1,10 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from tfw.networking.serialization import deserialize_all from tfw.networking import deserialize_all
from tfw.networking.event_handlers.server_connector import ServerConnector from tfw.networking.event_handlers import ServerConnector
class EventHandlerBase(ABC): class EventHandlerBase(ABC):
+4 -1
View File
@@ -1,8 +1,11 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from typing import List from typing import List
from transitions import Machine from transitions import Machine
from tfw.components.mixins.callback_mixin import CallbackMixin from tfw.components.mixins import CallbackMixin
class FSMBase(CallbackMixin): class FSMBase(CallbackMixin):
+9
View File
@@ -0,0 +1,9 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .serialization import decode_if_needed, encode_if_needed, serialize_all, deserialize_all
from .zmq_connector_base import ZMQConnectorBase
from .controller_connector import ControllerConnector
from .message_sender import MessageSender
from .event_handlers.server_connector import ServerUplinkConnector as TFWServerConnector
from .server.tfw_server import TFWServer
+4 -1
View File
@@ -1,8 +1,11 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import zmq import zmq
from zmq.eventloop.zmqstream import ZMQStream from zmq.eventloop.zmqstream import ZMQStream
from tfw.config import TFWENV from tfw.config import TFWENV
from tfw.networking.zmq_connector_base import ZMQConnectorBase from tfw.networking import ZMQConnectorBase
class ControllerConnector(ZMQConnectorBase): class ControllerConnector(ZMQConnectorBase):
@@ -0,0 +1,4 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .server_connector import ServerConnector, ServerUplinkConnector, ServerDownlinkConnector
@@ -1,11 +1,14 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from functools import partial from functools import partial
import zmq import zmq
from zmq.eventloop.zmqstream import ZMQStream from zmq.eventloop.zmqstream import ZMQStream
from tfw.networking.serialization import serialize_all from tfw.networking import serialize_all
from tfw.networking import ZMQConnectorBase
from tfw.config import TFWENV from tfw.config import TFWENV
from tfw.networking.zmq_connector_base import ZMQConnectorBase
class ServerDownlinkConnector(ZMQConnectorBase): class ServerDownlinkConnector(ZMQConnectorBase):
@@ -1,6 +1,9 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from datetime import datetime from datetime import datetime
from tfw.networking.event_handlers.server_connector import ServerUplinkConnector from tfw.networking.event_handlers import ServerUplinkConnector
class MessageSender: class MessageSender:
+3
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import json import json
+7
View File
@@ -0,0 +1,7 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from .event_handler_connector import EventHandlerConnector, EventHandlerUplinkConnector, EventHandlerDownlinkConnector
from .tfw_server import TFWServer
from .zmq_websocket_handler import ZMQWebSocketProxy
from .controller_responder import ControllerResponder
@@ -1,5 +1,7 @@
from tfw.networking.controller_connector import ControllerConnector # Copyright (C) 2018 Avatao.com Innovative Learning Kft.
from tfw.networking.serialization import deserialize_all, serialize_all # All Rights Reserved. See LICENSE file for details.
from tfw.networking import deserialize_all, serialize_all, ControllerConnector
class ControllerResponder: class ControllerResponder:
@@ -1,8 +1,10 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import zmq import zmq
from zmq.eventloop.zmqstream import ZMQStream from zmq.eventloop.zmqstream import ZMQStream
from tfw.networking.zmq_connector_base import ZMQConnectorBase from tfw.networking import ZMQConnectorBase, serialize_all
from tfw.networking.serialization import serialize_all
from tfw.config import TFWENV from tfw.config import TFWENV
from tfw.config.logs import logging from tfw.config.logs import logging
+10 -4
View File
@@ -1,11 +1,15 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
from collections import defaultdict from collections import defaultdict
from tornado.web import Application from tornado.web import Application
from tfw.networking.server.zmq_websocket_handler import ZMQWebSocketProxy from tfw.networking import MessageSender
from tfw.networking.event_handlers.server_connector import ServerUplinkConnector from tfw.networking.event_handlers import ServerUplinkConnector
from tfw.message_sender import MessageSender from tfw.networking.server import EventHandlerConnector
from tfw.config.logs import logging from tfw.config.logs import logging
from .zmq_websocket_handler import ZMQWebSocketProxy
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
@@ -16,10 +20,12 @@ class TFWServer:
self._fsm_updater = FSMUpdater(self._fsm) self._fsm_updater = FSMUpdater(self._fsm)
self._fsm_manager = FSMManager(self._fsm) self._fsm_manager = FSMManager(self._fsm)
self._fsm.subscribe_callback(self._fsm_updater.update) self._fsm.subscribe_callback(self._fsm_updater.update)
self._event_handler_connector = EventHandlerConnector()
self.application = Application( self.application = Application(
[(r'/ws', ZMQWebSocketProxy, {'make_response': self.make_response, [(r'/ws', ZMQWebSocketProxy, {'make_response': self.make_response,
'proxy_filter': self.proxy_filter})] 'proxy_filter': self.proxy_filter,
'event_handler_connector': self._event_handler_connector})]
) )
#self.controller_responder = ControllerResponder(self.fsm) TODO: add this once controller stuff is resolved #self.controller_responder = ControllerResponder(self.fsm) TODO: add this once controller stuff is resolved
@@ -1,19 +1,23 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import json import json
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from tornado.websocket import WebSocketHandler from tornado.websocket import WebSocketHandler
from tfw.networking.serialization import deserialize_all from tfw.networking import deserialize_all
from tfw.networking.server.event_handler_connector import EventHandlerConnector
from tfw.config.logs import logging from tfw.config.logs import logging
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
class ZMQWebSocketHandler(WebSocketHandler, ABC): class ZMQWebSocketHandler(WebSocketHandler, ABC):
_event_handler_connector = EventHandlerConnector()
instances = set() instances = set()
def initialize(self, **kwargs):
self._event_handler_connector = kwargs['event_handler_connector']
def prepare(self): def prepare(self):
ZMQWebSocketHandler.instances.add(self) ZMQWebSocketHandler.instances.add(self)
@@ -24,14 +28,13 @@ class ZMQWebSocketHandler(WebSocketHandler, ABC):
LOG.debug('WebSocket connection initiated') LOG.debug('WebSocket connection initiated')
self._event_handler_connector.register_callback(self.zmq_callback) self._event_handler_connector.register_callback(self.zmq_callback)
@classmethod def zmq_callback(self, msg_parts):
def zmq_callback(cls, msg_parts): keyhandlers = {'mirror': self.mirror}
keyhandlers = {'mirror': cls.mirror}
key, data = deserialize_all(*msg_parts) key, data = deserialize_all(*msg_parts)
LOG.debug('Received on pull socket: %s', data) LOG.debug('Received on pull socket: %s', data)
if key not in keyhandlers: if key not in keyhandlers:
for instance in cls.instances: for instance in ZMQWebSocketHandler.instances:
instance.write_message(data) instance.write_message(data)
else: else:
try: try:
@@ -39,10 +42,9 @@ class ZMQWebSocketHandler(WebSocketHandler, ABC):
except KeyError: except KeyError:
LOG.error('Invalid mirror message format! Ignoring.') LOG.error('Invalid mirror message format! Ignoring.')
@classmethod def mirror(self, data):
def mirror(cls, data):
key = data['key'] key = data['key']
cls._event_handler_connector.send_message({'data': data}, key) self._event_handler_connector.send_message({'data': data}, key)
def on_message(self, message): def on_message(self, message):
LOG.debug('Received on WebSocket: %s', message) LOG.debug('Received on WebSocket: %s', message)
@@ -62,10 +64,11 @@ class ZMQWebSocketHandler(WebSocketHandler, ABC):
class ZMQWebSocketProxy(ZMQWebSocketHandler): class ZMQWebSocketProxy(ZMQWebSocketHandler):
# pylint: disable=abstract-method # pylint: disable=abstract-method
def initialize(self, make_response, proxy_filter): def initialize(self, **kwargs):
# pylint: disable=arguments-differ # pylint: disable=arguments-differ
self._make_response = make_response super(ZMQWebSocketProxy, self).initialize(**kwargs)
self._proxy_filter = proxy_filter self._make_response = kwargs['make_response']
self._proxy_filter = kwargs['proxy_filter']
def on_message(self, message): def on_message(self, message):
message = json.loads(message) message = json.loads(message)
+3
View File
@@ -1,3 +1,6 @@
# Copyright (C) 2018 Avatao.com Innovative Learning Kft.
# All Rights Reserved. See LICENSE file for details.
import zmq import zmq
+1 -5
View File
@@ -1,5 +1,5 @@
[supervisord] [supervisord]
user=user user=root
logfile = /tmp/supervisord.log logfile = /tmp/supervisord.log
loglevel = debug loglevel = debug
pidfile = /tmp/supervisord.pid pidfile = /tmp/supervisord.pid
@@ -18,9 +18,5 @@ command=/usr/sbin/nginx -g 'daemon off;'
autostart=true autostart=true
autorestart=true autorestart=true
[program:app]
directory=%(ENV_TFW_APP_DIR)s
command=python3 app.py
[include] [include]
files=%(ENV_TFW_SUPERVISORD_COMPONENTS)s/*.conf files=%(ENV_TFW_SUPERVISORD_COMPONENTS)s/*.conf