miso

A framework to develop intuitive user interfaces for meduCA’s bioreactors and the next generation of hardware in iGEM.

Foreword

Bioreactors are, perhaps, the single most ubiquitous type of hardware in iGEM history. Each year, countless teams design purpose-built vessels to optimize and control the growth environments of biological organisms, each with a curated, unique combination of sensors and power devices. Photosynthetic Cyanobacteria, for instance, might require $CO_2$ sensors to control the gas composition inside the reactor, while E coli. might call for a combination of heating coils and agitator pumps to stimulate cell division. Due to the vast diversity in hardware design goals, no two teams’ firmware are identical, either; each program is typically tweaked to support the specific hardware implementation and requirements for which it was created. As a radical alternative, UBC iGEM proposes miso: a declarative, hardware-agnostic, and endlessly extensible firmware platform.

Introduction

Firmware refers to software that directly controls hardware devices such as sensors or motors, a critical aspect of designing novel hardware systems. Without firmware, it is impossible to control motor speed, direction, and timing; manage the activation and deactivation of heating systems; or ensure consistent polling of sensors.

miso is a firmware platform that enables all that and more, with user control offloaded to a web-based interface.

Key highlights

• miso powered our bioreactors and other hardware systems, enabling us to optimize cell culture conditions in Wet Lab
• miso acts as a platform that empowers teams in iGEM — as well as labs everywhere — to easily spin up a control interface for their purpose-built hardware
• miso leverages common standards and protocols (client-server architecture, RESTful APIs)
• miso is open-source and welcomes contributors to expand its standard library of firmware modules

Architecture

miso leverages a client-server architecture. Specifically:

• The miso server must be started on a Raspberry Pi microcontroller running a recent version of Raspberry Pi OS
  • This microcontroller will be connected to any and all hardware devices that you wish to control; it serves as the low-lever layer that communicates user directives to the hardware
  • The server opens a port on the local network, which clients can connect to
• Devices on the same local network can connect to this port, and become clients
  • The miso client web interface is forwarded by the server to the user’s device
  • A communications channel (implemented using the WebSocket API) is opened between the client and the server, enabling real-time control

User Reference

Raspberry Pi Setup/Installation

To use miso, you must obtain a Raspberry Pi. It should be set up as follows:

• Download Raspberry Pi Imager, and use it to flash a recent version of Raspberry Pi OS onto your device
  • Installing the headless version of the operating system (i.e., no graphical interface) is preferred; the rest of this guide will assume non-graphical use. It is also possible to flash the complete version of Raspberry Pi OS, connect a display to your Pi, and set up the Pi graphically.
  • When installing Raspberry Pi OS, make sure to enable SSH functionality and set it up to support the wireless network you wish to use (or, use Ethernet connectivity).
  • Set up a descriptive hostname for your device; this will be used later to connect clients.
• On another device, use SSH to start a shell session on your Pi
  • By default, Raspberry Pi OS supports Avahi-based mDNS resolution. This means you can SSH onto the Pi from another device by using the command ssh <hostname>.local, where <hostname> is the hostname you set in the previous step.
• Install miso:

cargo install --git https://gitlab.igem.org/2025/software-tools/ubc-vancouver miso

Configure Hardware

You can now connect wire any hardware devices that you wish to control to the Raspberry Pi’s GPIO pins.

Now, create a TOML file to declare the hardware that you would like to control.

TOML Schema

The TOML file MUST have two top-level tables, motors and sensors:

[sensors]
# ...

[motors]
# ...

The [sensors] table MAY contain sensor definitions. For example:

[sensors]
reactor_temp = { model = "DS18B20", pin = 5 }

See sensors for more information on valid sensor definitions.

The [motors] table MAY contain motor definitions. For example:

[motors]
impeller = { model = "DRV8825", step_pin = 20, dir_pin = 26, speed = 0, direction = "Backward" }

See motors for more information on valid motor definitions.

To assist users in defining valid configuration files, a schema is provided. Download the schema file and place it next to your TOML configuration file, then add the line

#:schema ./config_schema.json

to the top of your TOML config. Now, any editor that supports TOML language servers (e.g., taplo, tombi, …) will show diagnostics on invalid configurations. Alternatively, you can install jsonschema-for-toml:

cargo install jsonschema-for-toml

and use it to validate your config:

λ jsonschema-for-toml config_schema.json -i example_config.toml
example_config.toml - INVALID. Errors:
1. Additional properties are not allowed ('motor' was unexpected)
2. "motors" is a required property

Pins, Names, and A Philosophy

Fields in the [sensors] and [motors] tables provide options to configure which GPIO pins each device is connected to, and what name should be assigned to the device. For instance, when defining a temperature sensor:

reactor_temp = { model = "DS18B20", pin = 5 }

the field name (in this case reactor_temp) defines the display name of the hardware device, model defines what type of device is connected (in this case a DS18B20 digital thermometer), and pin defines which GPIO pin the data wire is connected to (in this case GPIO5).

This configuration philosophy makes it simple for users to define configurations for diverse hardware devices, by declaring different permutations of input and power devices.

Start the server

Once an adequate configuration has been defined, the miso server can be started. Simply run miso, passing your desired config file:

miso my_config.toml

This will spawn a web server on the local network, defaulting to 127.0.0.1:9999 as the IP address and port. This can be configured by using the IP and PORT environment variables:

IP="0.0.0.0" PORT=8888 miso my_config.toml

If your Pi does not have a graphical display, and you wish to control your hardware from other devices on the local network, IP=0.0.0.0 is typically desired. You can also forward this connection to the public interview using a service like ngrok, but that will not be covered in this guide.

Connect clients

On a different device, make sure you are connected to the same network as the Pi. Open your browser of choice, and navigate to http://<hostname>.local:<PORT>. For instance, if your Raspberry Pi’s hostname is “igempi” and you use the default port, you would navigate to http://igempi.local:9999. The miso client interface SHOULD now show up on your device, providing you an interface to control your configured hardware devices.

Web UI

Here is an example of the miso client UI:

it is divided into two primary sections, the sensor interface (above) and the motor interface (below).

Sensor Interface

This interface shows a graph of real-time sensor data, streamed directly from the server. A dropdown menu in the top-left allows users to toggle between configured sensors:

The Download button in the top-right enables users to inspect and download the raw sensor data as a CSV file, for external analysis or visualization:

Motor Interface

The dropdown menu in the top left allows users to swap the control interface between connected motors. A toggle switches motor direction on-click, and an input field allows users to set the motor speed with a value between 0 (off) and 100 (full speed). Hitting “Stop” immediately sets the speed to 0.

Developer Reference

The miso server and clients communicate via the following protocols:

• GET / or GET /index.html: serves the miso client UI
  • A WebSocket will be opened at /ws
• GET /csv: serves all sensor data as a CSV file Although miso provides a default client UI, the miso API is standardized, so additional client implementations are feasible. The miso server is implemented in the Rust programming language, and the miso client is implemented in TypeScript.

WebSocket Schema

The WebSocket at /ws is used to facilitate real-time communication between the server and client. All messages are sent in JSON, and MUST adhere to the following schema.

Initialization

On initial connection, a message will be sent from the server to the connected client:

{
  "motors": {
    "impeller": {
      "direction": "Backward",
      "speed": 0
    }
  },
  "sensors": {
    "reactor_temp": [
      {
        "data": 36.952172164840135,
        "timestamp": 0
      },
      {
        "data": 36.93485868768756,
        "timestamp": 1
      },
      {
        "data": 36.69193759642496,
        "timestamp": 2
      }
    ]
  },
  "type": "handshake"
}

The object SHALL contain the following fields:

• type, which SHALL be handshake for the initialization handshake
• motors, which SHALL contain zero or more motor names, corresponding to their initial parameters:
  • direction, which SHALL be Forward or Backward
  • speed, which SHALL be a number from 0 to 100
• sensors, which SHALL contain zero or more sensor names, corresponding to a list of data instances, each with the fields:
  • data, which contains the sensor read data
  • timestamp, which contains the timestamp at which the data was read

Updates

The same format MUST be used by:

• the server to send an update to all clients
• a client to send an update to the server with the only difference being that client-to-server updates SHOULD NOT contain a sensors field.

{
  "type": "update",
  "sensors": {
    "reactor-temperature": 27.2,
    "reactor-ph": 9.1
  }
}

{
  "type": "update",
  "motors": {
    "impeller": {
      "speed": 40
    }
  }
}

The object MUST contain the following fields:

• type, which MUST be update for updates
• motors, which MUST correspond to an object containing a motor property (speed or direction), along with the desired target value
• sensors, which SHALL contain an object that corresponds sensor names to a new read value
  • Note: the sensors field may not include an entry for every sensor defined in the initialization message; sensors will be skipped if the read failed

Defining firmware interfaces

Support for hardware devices is added in the firmware crate; sensors are added in crates/firmware/src/sensors and motors are added in crates/firmware/src/motors. Both of these modules contain a mod.rs file which defines a Sensor and Motor trait (a trait in Rust is like an interface in other languages). To define a new firmware interface:

• Create a new file in the module (e.g., crates/firmware/src/sensors/my_sensor.rs)
• Add it as a public module in the relevant mod.rs file (e.g., pub mod my_sensor in crates/firmware/src/sensors/mod.rs)
• Define a struct for your sensor or motor, and derive Serialize/Deserialize for it. For instance, in my_sensor.rs:

#[derive(Serialize, Deserialize)]
struct MySensor {
    pin: u8,
}

• Implement the Sensor or Motor trait for your sensor/motor, and tag the trait implementation with #[typetag::serde]:

#[typetag::serde]
impl Sensor for MySensor {
    #[cfg(feature = "hardware")]
    fn read(&self) -> Result<f64, ReadError> {
        todo!()
    }

    fn simulate(&self, history: Vec<&f64>, history_len: usize) -> Result<f64, ReadError> {
        todo!()
    }
}

Congratulations; you have defined a new firmware module for miso! It is immediately connected to the configuration API, and users can declare instances of MySensor in their TOML configuration:

[sensors]
very_special_sensor = { model = "MySensor", pin = 5 }

Miso Standard Library

miso provides a so-called “standard library” of pre-defined hardware devices. Currently, this includes all sensors and motors used in our hardware projects this year.

Sensors

DS18B20

A module for the DS18B20 digital temperature sensor.

Configuration fields

• pin: the data pin for the sensor

Data output

A temperature in degrees Celsius.

Motors

L298N

A module for the L298N DC motor driver.

Configuration fields

• pin_1: first direction pin
• pin_2: the second direction pin
• en_a: the speed pin; MUST be a PWM-enabled pin

Implementation details

This firmware module controls motor speed and direction using 3 GPIO pins: two digital pins (pin_1/pin_2) that set direction via H-bridge control (HIGH/LOW combinations), and one PWM (Pulse Width Modulation)-capable pin (en_a) that regulates speed through duty cycle modulation. Speed values (0-100) are scaled to PWM duty cycles, with special handling for zero speed that sets both direction pins to LOW to brake.

PWM setup

PWM is a technique for controlling power delivery by rapidly switching a signal between ON (HIGH) and (OFF) LOW states. PWM maintains a constant voltage but changes the ratio of time between ON and OFF to control the delivery.

PWM requires special OS-level setup on Raspberry Pis. See this documentation for additional details.

DRV8825

A module for the DRV8825 stepper motor driver.

Configuration fields

• m0_pin/m1_pin/m2_pin: pins which set step size
• step_pin: step power pin
• dir_pin: direction control pin

Implementation details

This implementation controls motor speed and direction using pulse/direction interface: one digital pin (dir_pin) to set direction and one digital pin (step_pin) that advances the motor on each rising edge. The speed of the motor is managed by adjusting the frequency of step pulses; the m0/m1/m2 pins determine the size of the step that the motor takes.

The motor speed is controlled by adjusting the frequency of pulses sent to the STEP pin. Each rising edge contributes to the motor by one step, therefore the effective speed is simply determined by how frequent each pulse is. To incorporate this in the Rust implementation, a background thread controls the pulses forwarded to the STEP pin; the amount of time it waits between each toggle is calculated from a desired speed.

The relationship is:

$$f_{\text{steps}} = \frac{\text{speed}_{\text{pct}}}{100} \times f_{\text{max}}$$

where:

• $speed_{pct}$ = The speed percentage (0-100)
• $f_{max}$ = Maximum step frequency (250KHz)
• $f_{steps}$ = The resulting step frequency applied to the motor

DBTLs and Development

Proof of Concept 1

Initially, we planned for the firmware platform to have four parts: hardware, database, server, and frontend. The hardware layer would directly read sensors and power motors, with the resulting sensor data being stored in the database layer for future analysis. The server layer would coordinate sending data from the hardware & database layers to a frontend web interface, allowing users to easily monitor and control the bioreactor in real time.

The hardware, database, and server layers would run directly on the bioreactor’s microcontroller. We selected Raspberry Pi chips for this purpose. Compared to similar systems like Arduinos, these chips provided us with a full Linux environment. This allowed us to use more convenient APIs for tasks like networking and sensor control.

To implement the firmware platform, we decided on a tech stack powered by fullstack Rust. The server layer would use a custom-build Rust library and host a WebAssembly (WASM) web frontend using the Dioxus framework. To communicate with electronics on the Pi, our hardware layer used the RPPAL library as an ergonomic abstraction over the chip’s peripherals. The database layer would write to a simple SQLite database on-chip.

We chose this tech stack with our goals of performance, maintainability, and flexibility in mind. Rust’s high baseline performance and strict safety checks would help us keep the firmware maintainable. Additionally, using Rust across the board would allow our team to be productive on all areas of the firmware with knowledge of a single language. Existing libraries for reading config files in formats like TOML along with powerful macros and types would help us implement the flexibility we were looking for.

Implementation

To test-drive our tech stack, we designed our first proof-of-concept implementation: a web interface with a button toggling an LED on our Raspberry Pi. Although this was a simple project, it allowed us to confirm if all the parts of our stack worked together properly. Furthermore, it enabled us to familiarize ourselves with the Rust package ecosystem for hardware control.

This PoC is implemented in this Git repository.

Proof of Concept 2

For our second proof-of-concept, we extended our previous work to send data to the web frontend, where it was persisted and plotted. This was much closer in implementation to how our final firmware works. Furthermore, we implemented multiple new APIs:

• Custom sensor definition DSL

defsensors! {
    sensor DS18B20(pin_number) {
        // definition here
    }

    sensor other(pin_number) {
        // definition here
    }
}

• TOML-based configuration (deserialization defined by defsensors! macro)

[sensors]
reactor_temp = { model = "DS18B20", pin = 5 }
reactor_ph   = { model = "PhSensor", pin = 4 }

Implementation

The defsensors macro was implemented via a Rust procedural macro. It would internally generate an enum with each sensor as a variant, along with an associated read function and a deserialization implementation. It was performant, avoiding the virtual function lookups at runtime associated trait objects. However, the macro was complex and difficult to extend, making it highly inflexible. For instance, when we wished to add simulation functionality to sensors, the entire syntax needed to be redesigned.

The web frontend was implemented by leveraging the Dioxus library. Dioxus is a library for fullstack Rust on the web, providing an ergonomic API to connect web and server components. However, it leverages web assembly (WASM) for its web frontend, making it difficult to interact with JavaScript libraries (i.e., for plotting). We wished to leverage Observable Plot for data visualization on the frontend, but we ran into consistent issues when vendoring the dependency (Dioxus would refuse to load it in the correct order).

A little bit of chaos

One of our primary learnings from PoC 1 is that RPPAL cannot build on non-Linux systems. As such, we designed a data simulation framework known as entropy which allowed us to simulate sensor data. On non-Linux systems, sensor read calls are automatically swapped out for simulate calls, allowing our dev team to test our various subsystems without functional sensors available.

entropy implements its data simulation through stochastic simulation; however, it internally implements a custom, fixed-size buffer that allows it to simulate realistic trends. Each data point is generated based on:

• Random drift
• “Momentum” over the last few data points
• Reversion to the expected value entropy is available as a standalone project here, and can be used for realistic trend simulation across different types of projects. Its public API provides numerous configuration options:

#[derive(Copy, Clone)]
pub struct SimulationConfig {
    starting_position: StartingPosition,
    drift_scale: f64,
    momentum_curve: CurveType,
    momentum_scale: f64,
    reversion_curve: CurveType,
    reversion_scale: f64,
    clamped: bool,
}

#[derive(Copy, Clone)]
pub enum StartingPosition {
    BottomThird,
    MiddleThird,
    TopThird,
}

#[derive(Copy, Clone)]
pub enum CurveType {
    Linear,
    Quadratic,
    Logarithmic,
}

This PoC is implemented at this Git repository.

The finale

The final implementation of miso saw the major design change of switching from a fullstack Rust framework to embracing a native web frontend.

jumpdrive!

We designed a custom from-scratch web framework known as known as jumpdrive. It was designed to be minimally-featured, but perfectly suit our use case.

jumpdrive! {
    dir = "../web",
    ws = "/ws": websocket_endpoint,
    routes = {
            "/csv": csv_endpoint
    },
    err = error_handler
}

Properties

• dir: a directory to serve
  • This directory is bundled at compile-time, and its contents are served at runtime via GET requests which correspond to relative paths.
  • For instance, the request GET /index.html HTTP/1.1 would cause the server to forward the contents of ../web/index.html.
• ws: a WebSocket path to spawn, and a callback function for new connections
• routes: a map of additional GET endpoints and their corresponding callback functions
• err: an error handling callback for network or protocol failures

Testing and Learning

Initially, when an incoming request is detected, jumpdrive would peek the request to identify its type (GET endpoint vs WebSocket endpoint), then either respond to the GET request or execute a WebSocket handshake to upgrade the stream. However, we noticed an issue; infrequently, GET endpoint requests would fail with a “reset connection” (RST) error, making it almost seem like the GET request was executing multiple times in rapid succession.

This turned out to be a difficult-to-catch bug in the library. jumpdrive works by binding a TCP listener to a socket address, and accepting incoming connections as TCP streams. When the incoming stream contains a GET request on a non-WebSocket endpoint, a response is sent and the stream is closed; if the stream comes through a WebSocket endpoint, the stream is upgraded and remains alive. It turns out the bug was caused by jumpdrive peeking requests on GET endpoints rather than consuming the incoming bytes; the relevant source code in the Linux kernel can be viewed here:

    /* As outlined in RFC 2525, section 2.17, we send a RST here because
     * data was lost. To witness the awful effects of the old behavior of
     * always doing a FIN, run an older 2.1.x kernel or 2.0.x, start a bulk
     * GET in an FTP client, suspend the process, wait for the client to
     * advertise a zero window, then kill -9 the FTP client, wheee...
     * Note: timeout is always zero in such a case.
     */
    if (unlikely(tcp_sk(sk)->repair)) {
        sk->sk_prot->disconnect(sk, 0);
    } else if (data_was_unread) {
        /* Unread data was tossed, zap the connection. */
        NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONCLOSE);
        tcp_set_state(sk, TCP_CLOSE);
        tcp_send_active_reset(sk, sk->sk_allocation,
                      SK_RST_REASON_TCP_ABORT_ON_CLOSE);

This is the source of the bug: the kernel was sending a RST signal to the stream since it believed that data was lost (the stream was not consumed). As such, a simple fix was implemented by flushing all remaining bytes in the socket after sending a response:

fn clear_socket(stream: &mut TcpStream) {
    let mut buf = [0; 2048];
    let _ = stream.read(&mut buf);
}

Note: the stream should never realistically contain more than 2048 incoming bytes.

Trait system

Sensor and motor definitions transitioned from a DSL approach to simple trait implementations. This significantly reduced their complexity while simultaneously improving their flexibility to potential behavioral extensions in future versions. Currently, these traits are defined as follows:

pub trait Sensor: Send + Sync {
    #[cfg(feature = "hardware")]
    fn read(&self) -> Result<f64, ReadError>;
    fn simulate(&self, history: Vec<&f64>, history_len: usize) -> Result<f64, ReadError>;
}


pub trait Motor: Send + Sync {
    #[cfg(feature = "hardware")]
    fn initialize(&mut self) -> MotorResult<()>;

    fn get_speed(&self) -> u64;
    fn set_speed(&mut self, speed: u64) -> MotorResult<()>;

    fn get_direction(&self) -> MotorDirection;
    fn set_direction(&mut self, dir: MotorDirection) -> MotorResult<()>;
}

Future Directions

std expansion

An ongoing area of development for miso involves expanding the sensor and motor implementations included in the standard library. During miso’s active development, these implementations were done on a need by need basis. However, now that miso server has stabilized, additional hardware implementations can be done at a more rapid pace.

Press and Play

What’s a motor anyways?

As such, miso is undergoing work to redefine the sensor/motor dichotomy as sensor/power instead, and move non-generic APIs (such as direction control) out of the core Motor trait and into an extension trait. The broad design for this is as follows:

#[typetag::serde(tag = "model")]
pub trait Power: Send + Sync {
    #[cfg(feature = "hardware")]
    fn initialize(&mut self) -> PowerResult<()>;

    fn get_power(&self) -> u64;
    fn set_power(&mut self, speed: u64) -> PowerResult<()>;

    fn as_directional(&self) -> Option<&dyn Directional> {
        None
    }
    fn as_directional_mut(&mut self) -> Option<&mut dyn Directional> {
        None
    }
}

The new as_directional and as_directional_mut APIs are the basis for how “extension behavior” will be defined; executing these methods on a Power object will return either Some(...) if the power object supports “direction” control, or None (default) if this behaviour is not supported. This will allow miso to support a broad range of power devices. Furthermore, the ability to call these methods as callbacks on certain sensor readings (e.g., activate a heating pad when temperature is too low, or disable it when temperature is too high) is being developed.