Zipper

Introduction

A zipper connects (or zips together) a continuous flow of data generated by source and stream function that operate on the data stream. To define a workflow configuration for zipper, you can simply create a .yaml file (see the example below).

YoMo uses the QUIC transport protocol, which largely improves the speed of data transfer, and supports the encoding/decoding of streams that have different data types through the y3 codec.

zipper

Configure and Run zipper

workflow.yaml

You can define a workflow configuration for zipper in a .yaml file. For example, here is the workflow of a simple real-time noise monitoring system:

# zipper/workflow.yaml
name: Service
host: localhost
port: 9000
functions:
  - name: Noise
  - name: MockDB
  • name: Name of the zipper.
  • host and port: zipper will be listening on host:port.
  • functions: A list of stream functions. In this example, Noise will monitor changes in the noise level and print out a warning message when it reaches a certain threshold. MockDB will save the data to a database.

To learn more about yaml, see Learn X in Y minutes, where X=yaml.

Running the Sample Program

1. Install YoMo CLI

Please refer to the instructions here.

2. Start zipper

yomo serve -c ./zipper/workflow.yaml

Using config file: ./zipper/workflow.yaml
2021/11/11 16:09:54 [yomo:zipper] [AddWorkflow] 0, Noise
ℹ️   Running YoMo-Zipper...
2021/11/11 16:09:54 [yomo:zipper] Listening SIGTERM/SIGINT...
2021/11/11 16:09:54 [core:server](name:Service) Listening on: 127.0.0.1:9000, QUIC: [v1 draft-29]

3. Start stream-fn

go run ./sfn/main.go

2021/11/11 16:11:05 [core:client] use credential: [None]
2021/11/11 16:11:05 [core:client] ❤️  [Noise] is connected to YoMo-Zipper localhost:9000

4. Start source

go run ./source/main.go

2021/11/11 16:12:01 [core:client] use credential: [None]
2021/11/11 16:12:01 [core:client] ❤️  [yomo-source] is connected to YoMo-Zipper localhost:9000
2021/11/11 16:12:01 [source] ✅ Emit {192.13399 1636618321242 localhost} to YoMo-Zipper
2021/11/11 16:12:01 [source] ✅ Emit {132.86566 1636618321547 localhost} to YoMo-Zipper
2021/11/11 16:12:01 [source] ✅ Emit {199.17604 1636618321851 localhost} to YoMo-Zipper

5. Results

stream-fn (i.e., Noise) monitors changes in the noise level and prints out a warning message when it reaches a certain threshold:

2021/11/11 16:12:01 >> [sfn] got tag=0x33, data={ 0x1.80449ap+07  0x17d0e0dbd5a 0x6c 0x6f 0x63 0x61 0x6c 0x68 0x6f 0x73 0x74}
2021/11/11 16:12:01 >> [sfn] got tag=0x33, data={ 0x1.09bb38p+07  0x17d0e0dbe8b 0x6c 0x6f 0x63 0x61 0x6c 0x68 0x6f 0x73 0x74}
2021/11/11 16:12:01 >> [sfn] got tag=0x33, data={ 0x1.8e5a22p+07  0x17d0e0dbfbb 0x6c 0x6f 0x63 0x61 0x6c 0x68 0x6f 0x73 0x74}
Edit this page on GitHub Updated at Sat, May 28, 2022