Merge branch 'fsm_as_eventhandler'

README.md

@@ -93,7 +93,7 @@ To do this simply issue `BASEIMAGE_ONLY=version bash -c "$(curl -fsSL https://gi
 The repository of a tutorial-framework based challenge is quite similar to a regular challenge.
 The project root should look something like this:
 
-```text
+```
 your_repo
 ├── solvable
 │   └── [TFW based Docker image]
@@ -112,7 +112,7 @@ From now on we are going to focus on the `solvable` image.
 
 Let us take a closer look on `solvable`:
 
-```text
+```
 solvable
 ├── Dockerfile
 ├── nginx        webserver configurations
@@ -134,7 +134,7 @@ This means that in order to listen on more than a single port we must use a reve
 Any `.conf` files in `solvable/nginx/` will be automatically included in the nginx configuration.
 In case you want to serve a website or service you must proxy it through `TFW_PUBLIC_PORT`.
 This is really easy: just create a config file in `solvable/nginx/` similar to this one:
-```text
+```
 location /yoururl {
         proxy_pass http://127.0.0.1:3333;
 }
@@ -157,7 +157,7 @@ You can even configure your processes to start with the container by including `
 
 To run your own webservice for instance you need to create a config file in `solvable/supervisor/` similar to this one:
 
-```text
+```
 [program:yourprogram]
 user=user
 directory=/home/user/example/
@@ -183,17 +183,12 @@ Refer to the example in this repo.
 This folder contains the source code of a server running TFW and an other server running our event handlers.
 Note that this is not a part of the framework by any means, these are just simple examples.
 
-```text
-solvable/src
-├── tfw_server.py           tutorial-framework server
-├── event_handler_main.py   event handlers implemented in python
-└── test_fsm.py             example FSM
 ```
-
+solvable/src
-The core of the framework is the `TFWServer` class, which is instanciated in `tfw_server.py`.
+├── event_handler_main.py   event handlers implemented in python
-This class handles the forwarding of the messages from the frontend to the event handlers connecting to it via ZMQ.
+├── test_fsm.py             example FSM in python
-It also manages the FSM.
+└── test_fsm.yml            example FSM in yaml
-As you can see this file is set up to start with the container in `solvable/supervisor/tfw_server.conf`.
+```
 
 `event_handler_main.py` contains example usage of our pre-defined event handlers written in Python3.
 As you can see they run in a separate process (set up in `solvable/supervisor/event_handler_main.conf`).
@@ -202,14 +197,46 @@ These event handlers could be implemented in any language that has ZMQ bindings.
 Note that you don't have to use all our event handlers.
 Should you want to avoid using a feature, you can just delete the appropriate event handler from `event_handler_main.py`.
 
+`test_fsm.yml` and `test_fsm.py` are the implementations of the same FSM in YAML and Python to provide you examples of creating your own machine.
+
 It is genarally a good idea to separate these files from the rest of the stuff in `solvable`, so it is a good practice to create an `src` directory.
 
+### FSM
+
+A good state machine is the backbone of a good TFW challenge.
+
+There are two ways to define a state machine:
+    - Using a YAML configuration file
+    - Implementing it in Python by hand
+
+The first option allows you to handle FSM callbacks and custom logic in any programming language (not just Python) and is generally really easy to work with (you can execute arbitrary shell commands on events).
+You should choose this method unless you have good reason not to.
+This involves creating your YAML file (see `test_fsm.yml` for an example) and parsing it using our `YamlFSM` class (see `event_handler_main.py` for an example).
+
+The second option allows you to implement your FSM in Python, using the transitions library.
+To do this just subclass our `FSMBase` class or use our `LinearFSM` class for simple machines (see `test_fsm.py` for an example).
+
+In your FSM you can define callbacks for states and transitions.
+State callbacks:
+    - `on_enter`
+    - `on_exit`
+Transition callbacks:
+    - `before`
+    - `after`
+
+In your YAML file you can use these in the state and transition objects as keys, then add a shell command to run as a value (again, see `test_fsm.yml` for examples).
+
+It is also possible to add preconditions to transitions.
+This is done by adding a `predicates` key with a list of shell commands to run.
+If you do this, the transition will only succeed if the return code of all predicates was `0` (as per unix convention for success).
+
 ## Baby steps
 
 When creating your own challenge the process should be the following:
 1. Use our install script to bootstrap your dev environment
 2. Create an FSM that describes your challenge
-    - An example is in `solvable/src/test_fsm.py`
+    - An example is in `solvable/src/test_fsm.yml`
+    - The same FSM in python is in `solvable/src/test_fsm.py`
 3. Create a `TFWServer` instance and set it up to run:
     - Create a server app: `solvable/src/tfw_server.py`
     - Set it up to run: `solvable/supervisor/tfw_server.conf`

solvable/Dockerfile

@@ -7,13 +7,13 @@ RUN pip3 install Flask==1.0        \
                  git+https://github.com/avatao-content/tfwconnector.git#subdirectory=python3
 
 # Define variables to use later
-ENV TFW_SERVER_DIR="/srv/.tfw"                      \
+ENV TFW_EHMAIN_DIR="/srv/.tfw_builtin_ehs"          \
     TFW_WEBSERVICE_DIR="/srv/webservice"            \
     TFW_IDE_WD="/home/${AVATAO_USER}/workdir"       \
     TFW_TERMINADO_WD="/home/${AVATAO_USER}/workdir"
 
 # Copy TFW related stuff to a dedicated directory
-COPY solvable/src ${TFW_SERVER_DIR}/
+COPY solvable/src ${TFW_EHMAIN_DIR}/
 
 # Copy webservice to a dedicated directory
 COPY solvable/src/webservice/ ${TFW_WEBSERVICE_DIR}/

solvable/src/event_handler_main.py

@@ -1,15 +1,20 @@
 from ast import literal_eval
+from functools import partial
 
 from tornado.ioloop import IOLoop
 
+from tfw import YamlFSM
 from tfw.components import IdeEventHandler, TerminalEventHandler
 from tfw.components import ProcessManagingEventHandler, BashMonitor
 from tfw.components import TerminalCommands, LogMonitoringEventHandler
+from tfw.components import FSMManagingEventHandler
 from tfw.networking import MessageSender, TFWServerConnector
 from tfw.config import TFWENV
 from tfw.config.logs import logging
 from tao.config import TAOENV
 
+from test_fsm import TestFSM
+
 LOG = logging.getLogger(__name__)
 
 
@@ -76,6 +81,10 @@ class TestCommands(TerminalCommands):
 
 
 if __name__ == '__main__':
+    fsm = FSMManagingEventHandler( # TFW FSM
+        key='fsm',
+        fsm_type=partial(YamlFSM, 'test_fsm.yml')
+    )
     ide = IdeEventHandler(  # Web IDE backend
         key='ide',
         allowed_directories=[TFWENV.IDE_WD, TFWENV.WEBSERVICE_DIR],
@@ -96,7 +105,7 @@ if __name__ == '__main__':
         process_name='webservice',
         log_tail=2000
     )
-    eventhandlers = {ide, terminal, processmanager, logmonitor}
+    eventhandlers = {fsm, ide, terminal, processmanager, logmonitor}
 
     commands = TestCommands(bashrc=f'/home/{TAOENV.USER}/.bashrc')
     terminal.historymonitor.subscribe_callback(commands.callback)

solvable/src/test_fsm.py

@@ -1,3 +1,7 @@
+# This defines an FSM equvivalent to test_fsm.yml
+
+from os.path import exists
+
 from tfw import LinearFSM
 from tfw.networking import MessageSender
 
@@ -5,24 +9,29 @@ from tfw.networking import MessageSender
 class TestFSM(LinearFSM):
     # pylint: disable=unused-argument
 
-    def __init__(self, number_of_steps):
+    def __init__(self):
-        super().__init__(number_of_steps)
+        super().__init__(6)
         self.message_sender = MessageSender()
+        self.subscribe_predicate('step_3', self.step_3_allowed)
+
+    @staticmethod
+    def step_3_allowed():
+        return exists('/home/user/workdir/allow_step_3')
 
     def on_enter_1(self, event_data):
-        self.state_notify(1)
+        self.message_sender.send('FSM', 'Entered state 1!')
 
     def on_enter_2(self, event_data):
-        self.state_notify(2)
+        filename = '/home/user/workdir/cat.txt'
+        with open(filename, 'w') as ofile:
+            ofile.write('As you can see it is possible to write arbitrary python code here.')
+        self.message_sender.send('FSM', f'Entered state 2! Written stuff to {filename}')
 
     def on_enter_3(self, event_data):
-        self.state_notify(3)
+        self.message_sender.send('FSM', 'Entered state 3!')
 
     def on_enter_4(self, event_data):
-        self.state_notify(4)
+        self.message_sender.send('FSM', 'Entered state 4!')
 
     def on_enter_5(self, event_data):
-        self.state_notify(5)
+        self.message_sender.send('FSM', 'Entered state 5!')
-
-    def state_notify(self, state):
-        self.message_sender.send('TestFSM', f'Entered state {state}!')

solvable/src/test_fsm.yml

@@ -0,0 +1,43 @@
+# This defines an FSM equvivalent to test_fsm.py
+
+states:
+    -   name: '0'
+    -   name: '1'
+        on_enter: |
+            python3 -c "from tfwconnector import MessageSender; MessageSender().send('FSM', 'Entered state 1!')"
+    -   name: '2'
+        on_enter: |
+            file=/home/user/workdir/cat.txt
+            echo "As you can see it is possible to execute arbitrary shell commands here." >> $file
+            python3 -c \
+            "
+            from tfwconnector import MessageSender
+            MessageSender().send('FSM', 'Entered state 2! Written stuff to $file')
+            "
+    -   name: '3'
+        on_enter: |
+            python3 -c "from tfwconnector import MessageSender; MessageSender().send('FSM', 'Entered state 3!')"
+    -   name: '4'
+        on_enter: |
+            python3 -c "from tfwconnector import MessageSender; MessageSender().send('FSM', 'Entered state 4!')"
+    -   name: '5'
+        on_enter: |
+            python3 -c "from tfwconnector import MessageSender; MessageSender().send('FSM', 'Entered state 5!')"
+transitions:
+    -   trigger: step_1
+        source: '0'
+        dest: '1'
+    -   trigger: step_2
+        source: '1'      
+        dest: '2'
+    -   trigger: step_3
+        source: '2'
+        dest: '3'
+        predicates:
+            - '[ -f /home/user/workdir/allow_step_3 ]'  # in bash -f means that the file exists
+    -   trigger: step_4
+        source: '3'
+        dest: '4'
+    -   trigger: step_5
+        source: '4'
+        dest: '5'

solvable/src/tfw_server.py

@@ -1,17 +0,0 @@
-from functools import partial
-
-from tornado.ioloop import IOLoop
-
-from tfw.networking import TFWServer
-from tfw.config import TFWENV
-from tfw.config.logs import logging
-from test_fsm import TestFSM
-
-LOG = logging.getLogger(__name__)
-
-
-if __name__ == '__main__':
-    FiveStepTestFSM = partial(TestFSM, 5)
-    TFWServer(FiveStepTestFSM).listen(TFWENV.WEB_PORT)
-
-    IOLoop.instance().start()

solvable/supervisor/event_handler_main.conf

@@ -1,4 +1,4 @@
 [program:event_handler_main]
 user=root
-directory=%(ENV_TFW_SERVER_DIR)s
+directory=%(ENV_TFW_EHMAIN_DIR)s
 command=python3 event_handler_main.py

solvable/supervisor/tfw_server.conf

@@ -1,4 +0,0 @@
-[program:tfwserver]
-user=root
-directory=%(ENV_TFW_SERVER_DIR)s
-command=python3 tfw_server.py